Sou Nobukawa
Aya Shirama
Tetsuya Takahashi
Shigenobu Toda- 1Department of Computer Science, Chiba Institute of Technology, Narashino, Chiba, Japan
- 2Graduate School of Information and Computer Science, Chiba Institute of Technology, Narashino, Chiba, Japan
- 3Research Center for Mathematical Engineering, Chiba Institute of Technology, Narashino, Chiba, Japan
- 4Department of Preventive Intervention for Psychiatric Disorders, National Institute of Mental Health, National Center of Neurology and Psychiatry, Tokyo, Japan
- 5Research Center for Child Mental Development, Kanazawa University, Kanazawa, Japan
- 6Department of Neuropsychiatry, University of Fukui, Fukui, Japan
- 7Uozu Shinkei Sanatorium, Uozu, Toyama, Japan
- 8Department of Psychiatry, Shizuoka Psychiatric Medical Center, Shizuoka, Japan
- 9Department of Psychiatry and Behavioral Science, Kanazawa University, Kanazawa, Japan
- 10Department of Psychiatry, Showa University, Tokyo, Japan
Recent studies focusing on neural activity captured by neuroimaging modalities have provided various metrics for elucidating the functional networks and dynamics of the entire brain. Functional magnetic resonance imaging (fMRI) can depict spatiotemporal functional neural networks and dynamic characteristics due to its excellent spatial resolution. However, its temporal resolution is limited. Neuroimaging modalities such as electroencephalography (EEG) and magnetoencephalography (MEG), which have higher temporal resolutions, are utilized for multi-temporal scale and multi-frequency-band analyzes. With this advantage, numerous EEG/MEG-bases studies have revealed the frequency-band specific functional networks involving dynamic functional connectivity and multiple temporal-scale time-series patterns of neural activity. In addition to analyzing neural data, the examination of behavioral data can unveil additional aspects of brain activity through unimodal and multimodal data analyzes performed using appropriate integration techniques. Among the behavioral data assessments, pupillometry can provide comprehensive spatial-temporal-specific features of neural activity. In this perspective, we summarize the recent progress in the development of metrics for analyzing neural data obtained from neuroimaging modalities such as fMRI, EEG, and MEG, as well as behavioral data, with a special focus on pupillometry data. First, we review the typical metrics of neural activity, emphasizing functional connectivity, complexity, dynamic functional connectivity, and dynamic state transitions of whole-brain activity. Second, we examine the metrics related to the time-series data of pupillary diameters and discuss the possibility of multimodal metrics that combine neural and pupillometry data. Finally, we discuss future perspectives on these multiple and multimodal metrics.
1 Introduction
Mounting evidence from recent studies focusing on neural activities captured by neuroimaging modalities has provided diverse metrics for elucidating functional networks and dynamics in the entire brain [reviewed in Sporns and Seguin et al. (1, 2)]. The analysis of functional magnetic resonance imaging (fMRI) data to ascertain functional connectivity facilitates the elucidation of the functional whole brain network, called the “functional connectome” (3–5), owing to its fine spatial resolution for the brain, including the deep-located nuclei. Moreover, functional networks show dynamic (called dynamic functional connectivity) rather than static characteristics in response to cognitive and perceptual stimuli, even during the resting state (6–8) [reviewed in Preti et al. (9)]. This dynamic organization of functional connectivity is strongly linked to the complexity of local regional neural activity (10). Brain function is a representative “emergence” phenomenon produced by the integration of hierarchical and mutual interactions of neural activities of the brain (11–13). Notably, the broad-range mutual interactions among neural activities enable cognitive functions as the most complex emergent phenomena, venturing beyond mere signal propagation between specific regions [as reviewed in Thiebaut de Schotten and Forkel (14)]. To elucidate this mechanism, the use of comprehensive metrics of whole-brain spatiotemporal neural activities has gained widespread recognition (1).
fMRI can depict spatiotemporal functional neural networks and dynamic characteristics due to its excellent spatial resolution. However, its temporal resolution is limited ≲1 Hz. This prevents the elucidation of neural activity characteristics over a wide frequency range including delta, theta, alpha, beta, gamma, and high gamma ≲100 [Hz] and moment-to-moment dynamics (15). Neuroimaging modalities such as electroencephalography (EEG) and magnetoencephalography (MEG) with higher time resolutions have been utilized for multi-temporal scale and multi-frequency-band analyzes [reviewed in Beppi et al., Iivanainen et al., and Niso et al. (16–18)]. The vast collection of EEG/MEG data in these studies revealed the frequency-band-specific functional network involving dynamic functional connectivity and multiple temporal-scale time-series patterns of neural activities. This was captured as “complexity,” which refers to the degree of irregularity observed in time-series data of neural activity, evaluated using metrics from non-linear time-series analysis, including chaos theory, fractal analysis, and various types of entropy measures; additionally, moment-to-moment dynamic state transitions based on whole-brain neural activity were analyzed to capture temporal dynamics (19–22). However, these neuroimaging modalities have limitations related to volume conduction, which refers to the spurious synchronization of regional neural activities (23, 24). By focusing on the phase components of neural activities, this influence can be curtailed and typified as the phase lag index (PLI) and weighted PLI (25–29). Subsequently, the combinations of high-density EEG and MEG data and complementary cortical source localization techniques have significantly augmented the spatial resolution, yielding sustained high temporal resolutions (30–33). Moreover, considering the assembly of functional connectivity as a topological feature enables assessment of the functional networks among widely distributed brain regions [reviewed in Farahani et al., Ismail and Karwowski, Pegg et al. (34–36); see the typical example of Figure 1A).
Figure 1. (A) Typical examples of metrics as a topological feature of functional networks. Small-worldness based on the ratio of the clustering coefficient and path length, which represents the segregation and efficiency of signal propagation within the functional networks. Hub characteristics such as node degree (edge numbers in the node) and betweenness centrality (number of shortest paths passing through the node), which are effective for evaluating integration and propagation within functional networks. (B) Spatial-temporal objective domain of representative neuroimaging, including functional magnetic resonance imaging (fMRI), electroencephalography (EEG), and magnetoencephalography (MEG), pupillometry.
In addition to the analysis of neural data, the examination of behavioral data, such as heart rate, body movements, eye movements, and pupil behaviors, can reveal additional characteristics of brain activity through unimodal [reviewed in Lujan et al., Mahanama et al., and Gullett et al. (37–39)] and multimodal data analyses with appropriate integration techniques [reviewed in Iwama et al. (40)]. This encompasses daily fluctuations in neural activity, known as the circadian rhythm [reviewed in Foster (41)], the balance between neural systems in whole-brain activity, such as the sympathetic and parasympathetic neural systems [reviewed in Hyun and Sohn (42)], and the activity patterns of deep-located neural nuclei [reviewed in Aston-Jones and Cohen and Viglione et al. (43, 44)]. Amongst these behavioral data, pupillometry can provide comprehensive spatial-temporal-specific features of neural activity (45–47). Figure 1B illustrates the relationship between the depth of signal-source location in the brain and the major frequency components of the time-series data obtained by each neuroimaging modality. Pupillometry detects relatively high-frequency components of neural activity, ≲ several Hz, in deep-located brain regions; fMRI and EEG/MEG cannot capture such high-frequency components and the neural activities of deep brain regions (45, 48). Leveraging this specificity can pave the way for understanding the mechanisms underlying brain functions and introduce a novel approach for identifying dysfunctions responsible for psychiatric disorders (49). Effectively ascertaining the dynamic patterns of neural activity using pupillometry is beset by challenges when extremely high-dimensional brain activities are projected onto two-dimensional time series of pupil diameters in the left and right eyes through signal propagation with nonlinear transformation and diverse interactions of inherent neural activities (50, 51). Therefore, developing metrics to estimate the internal neural activity from the time-series data of pupil diameters is important, as evidenced by the substantial focus of recent research (49). Moreover, although many recent multimodal studies focused primarily on pupil size (52, 53), there have been increasing attempts to capture innate neural activity based on pupil dynamics beyond simple size measurements (51, 54–56).
In this perspective, we summarize the recent progress in developing metrics for analyzing neural data obtained from neuroimaging modalities such as fMRI, EEG, and MEG as well as behavioral data, with a focus on pupillometry data. First, we review the typical metrics of neural activity, emphasizing functional connectivity, complexity, dynamic functional connectivity, and dynamic state transitions of whole-brain activity. Second, we examine the metrics related to the time-series data of pupil diameters. Third, we discuss the possibility of multimodal metrics that combine neural and pupillometry data. Finally, we discuss future perspectives on these multiple and multimodal metrics.
2 Analysis of neural data
In the classical interpretation of cognitive functions, individual brain regions are viewed as modular systems, each playing a specific cognitive role. Information processed in each region is subsequently propagated through hierarchical neural pathways. However, recent neuroimaging studies have revealed that cognitive functions are the result of interactions among widely distributed brain regions, rather than mere signal propagation between specific regions [reviewed in Thiebaut de Schotten and Forkel (14)].
To elucidate local and regional neural activities, the levels of blood oxygen level-dependent (BOLD) signals in fMRI and the power components of EEG/MEG signals continue to be widely utilized, serving as the initial foundational methods. Moreover, functional connectivity is considered the most representative approach to evaluate signal propagation and the integration of brain activities (3–5). Initially, this evaluation employs correlation coefficients and coherence measures based on cross-spectrum analysis, which are used in fMRI. However, for EEG/MEG signals, the influence of volume conduction, which causes spurious synchronization due to the observation of the same signal source at different sensors on the scalp, degrades the spatial resolution of functional networks (23, 24). This influence can be mitigated by metrics that focus on phase components, such as the PLI (57) and the weighted PLI, an enhanced iteration of the PLI (27). By leveraging the high temporal resolution of EEG/MEG combined with these improvements, these metrics have been utilized to delineate frequency-band-specific functional networks related to aging (58), various cognitive functions (59), and psychiatric disorders (60, 61).
In addition to evaluating functional connectivity based on mere pairwise neural activities, assessing the interactions of neural activity at the whole-brain level entails considering the assembly of functional connectivity as a topological feature of functional networks [see the overview of typical example of network measures in Figure 1A; reviewed in Farahani et al., Ismail and Karwowski, and Pegg et al. (34–36)]. The clustering coefficient, path length, and their ratio, which are measures of small-worldness, represent the segregation and efficiency of signal propagation within the functional networks (62–64). Hub characteristics such as node degree and betweenness centrality (65) are effective for evaluating the integration and propagation within these networks. These metrics can capture aspects of functional network topology in cognitive processes (32), their abilities (66), and psychiatric disorders (67, 68). These topological features reflect the complexity of time-series data pertaining to local and regional activities. Specifically, the hub region of the brain is influenced by other brain regions during the integration process; this interaction induces region-specific complexity (58, 69, 70). This characteristic is significantly advantageous from an application viewpoint because the local and regional complexity can be evaluated using low-density EEG equipment, which offers high versatility. High-density EEG is required for evaluating the topological features of functional networks (71).
In the interactions among brain regions, the evaluation of functional networks based on averages obtained over long time-windows, known as static functional connectivity, is not sufficient. The interaction of neural activity during cognitive processes, even during the resting state, exhibits significant dynamic properties [reviewed in Hutchison et al. (72)]. Dynamic functional connectivity, which involves the temporal variation of functional connectivity through sliding windows, was proposed to evaluate dynamic interactions [reviewed in Hutchison et al. (72)]. This approach to dynamic functional connectivity has succeeded in capturing the network dynamics that support cognitive functions (73, 74) and alterations caused by psychiatric disorders (75, 76). Historically, there was ambiguity in setting the window, but currently, by associating it with the distribution of quasi-stable states of neural activity, a strict window setting can be achieved, allowing this method to successfully detect network patterns that reflect cognitive processes (77). Another recent advancement in studies on dynamic functional connectivity is the focus on dynamic phase synchronization instead of mere synchronization. This emphasizes the patterns produced by the phase difference in instantaneous phase components of neural activities between brain regions, and has successfully detected age-related alterations in functional networks, representing a significant improvement in detection abilities (78). Dynamic phase synchronization has evolved into a novel method for detecting dynamic whole-brain activity states based on the whole-brain distribution of instantaneous frequency, termed instantaneous frequency micro-states (79).
Considering future research trends for metrics of neural data, novel metrics are needed instead of focusing solely on the synchronization of neural activity. This is because the neural activity of the brain is dynamic in nature, including transient behaviors (79, 80). Therefore, the assumption of time invariance during certain periods in the evaluation of synchronization proves difficult in the presence of moment-to-moment dynamic behaviors. A recent study revealed that the propagation of a momentary drop in complexity within local and regional neural activity modulates interactions in hierarchical neural networks (10). This implies that moment-to-moment dynamic characteristics, which cannot be captured by synchronization alone, are essential for precise interactions related to brain activity. Metrics that address this challenge are likely to be developed in the future.
3 Analysis of pupil data
3.1 Metrics for pupillometry
In 1982, Usui and Stark demonstrated an inverted U-shape profile of the temporal complexity of pupil diameter characteristics relative to the degree of dilation (50). They attributed this phenomenon to autonomous fluctuations in internal neural activity and the non-linearity of neural pathways that control pupil diameter in response to the levels of activity. This understanding was based on the erstwhile knowledge that pupil diameters are regulated by the dilator muscle controlled by the sympathetic neural system and the sphincter muscle controlled by the parasympathetic neural system. Recently, it has been discovered that the autonomous neural fluctuations originate from the locus coeruleus (LC), which serves mainly as a common source for the sympathetic and parasympathetic pathways, at least partially (81) (see the overview of these neural pathways in Figure 2A). The LC plays a crucial role in coordinating the arousal and attention functions [reviewed in Aston-Jones and Cohen (43)]. Specifically, the inhibitory projections from the LC to the Edinger-Westphal nucleus (EWN) include contralateral projections on the left and right sides, and not only ipsilateral projections. This is unlike the sympathetic neural pathway, which comprises only ipsilateral projections (82). Furthermore, the LC, other nuclei, and various brain regions are sources of fluctuations in neural activity that significantly influence pupil diameter behaviors, depending on the cognitive process. This influence has been demonstrated by multimodal experiments that combined pupillometry with fMRI (46, 83). These characteristics of pupil diameters involve neural activity deep within the brain that cannot be captured by EEG/MEG (43) and faster components (45, 48) that cannot be detected by fMRI. Therefore, the addition of pupillometry is crucial for capturing deep brain interactions. However, pupil data present unique challenges due to missing data and artifacts from gaze movements and blinks, as well as nonlinear transformations and multiple projections from various brain regions and nuclei. Despite a substantial body of research proposing fundamental preprocessing, normalization, decomposition techniques, and temporal lag correction for pupil response [as illustrated in Figure 2B and reviewed in Fink et al. and Shirama et al. (84, 85)], evaluating such complex pupil data remains a challenging process.
Figure 2. (A) Neural pathways for regulating pupil diameter: the dilator muscle controlled by the sympathetic neural system and the sphincter muscle governed by the parasympathetic neural system. (B) Typical preprocessing pipeline for pupil data, including artifact removal, normalization, and decomposition.
In addition to assessing pupil size, which is the focus of the majority of studies (43, 46, 83, 86), the application of non-linear dynamic analysis has also progressed (51, 54–56). Recent studies have identified deterministic properties of pupil behavior that reflect internal neural activity (51, 55). Nobukawa et al. revealed the deterministic properties of spontaneous pupil behavior by combining sample entropy and surrogate data analyzes. Sviridova et al. also demonstrated these deterministic properties using various types of nonlinear analyzes, such as metrics from recurrence plots, the largest Lyapunov exponent, and deterministic nonlinear prediction (55). These properties, characterized by the complexity and symmetry of the left and right pupil diameters, are indicative of sleepiness (54) and deficits in attentional function (49, 56). Specifically, the complexity measured by Shannon entropy and determinism in cross-recurrence analysis correlates with sleepiness (54). Asymmetry, measured by transfer entropy between the left and right time series of pupil size, and differences in complexity between the right and left eyes measured by sample entropy, reflect the pathology of attention deficit hyperactivity disorder [ADHD; (49, 56)]. These metrics, including pupil size, complexity, and symmetry, have a complementary relationship in identifying pathologies related to attention deficits (49); therefore, they may be utilized to derive components of neural activity from multiple brain regions and nuclei.
3.2 Computational models for pupil data
Modeling that incorporates non-linear neural pathways and internal neural activity reproduced by non-linear dynamic systems is effective as a complementary approach for estimating internal neural activity from pupil characteristics (50, 51). A pioneering study in this field by Usui and Stark applied fluctuating neural activity to the sympathetic (to the pupil dilator muscle) and parasympathetic pathways (to the pupil sphincter muscle) (50). In this model, the non-linear characteristics of these neural pathways elicit the emergence of detailed time-series patterns of internal neural activity near threshold levels. Consequently, the degree of temporal complexity of pupil size exhibits an inverted U-shaped relationship with the degree of dilation. This characteristic aligns well with the actual temporal behaviors of pupil size. Johansson and Balkenius developed a more complex model that considers multiple projections related to pupil properties and a neural system comprising the amygdala, LC, cerebellum, and other regions (87). Additionally, Nobukawa's model, which includes precise mapping of both contralateral and ipsilateral projections from the LC to the EWN, demonstrates that contralateral projections play a significant role in enhancing the inverted-U shaped profile of complexity for the pupillary characteristics relative to the pupil size (51). This model also successfully estimated the imbalance of internal LC activity between the left and right sides in cases of ADHD (56). Poynter suggested that pupil diameter asymmetry could reflect a left-right imbalance in LC activity, and that this degree of asymmetry is correlated with the severity of inattention, impulsivity, and hyperactivity (88). Nobukawa's model identified that hyperactivity on the right side of the LC causes the pupil diameter asymmetry observed in patients with ADHD, as demonstrated via comparisons with physiological pupil behaviors (56).
Thus, the combination of multiple metrics for pupil behavior and the application of models to neural systems controlling pupil behavior can enhance the spatiotemporal utility for estimating inherent neural activity. It can also provide complementary data obtained from neural activities and facilitate the effective integration of neural and behavioral data.
4 Multimodal analysis for neural and pupillometry sata
Recent studies utilizing simultaneous multimodal measurements of neural and behavioral activities, including pupil diameter, heart rate, and body movements, have demonstrated strong correlations between dynamic neural processes across various hierarchical levels, from single-neuron spikes to global brain networks, and these behavioral metrics (89–91) [see reviews in Aston-Jones and Cohen and van der Wel and van Steenbergen (43, 92)]. Specifically, the dynamic characteristics of pupil size offer valuable insights into neural states, reflecting factors such as arousal levels (93), cognitive functions (94, 95), and psychiatric conditions (49, 96–98) [reviewed in Aston-Jones and Cohen and van der Wel and van Steenbergen (43, 92)] (51).
As discussed in Section 2, various metrics have been developed to capture neural network activities. In parallel, research integrating multimodal measurements, such as the simultaneous monitoring of pupil dynamics with neuroimaging, has advanced significantly (52, 53, 99). As illustrated in Figure 1B, this approach seeks to utilize the spatial and temporal coverage of pupil measurements to complement the spatiotemporal range of EEG, MEG, and fMRI. Specifically, studies have highlighted correlations between pupil size and steady-state response BOLD signals obtained from fMRI, as well as associations with the selective attention, salience, error-detection, and decision-making processes (52). Furthermore, functional connectivity analyzes have revealed correlations between the functional connections of the superior frontal gyrus and pupil size, suggesting that these measurements can provide insights beyond regional activity alone (52).
Despite these advances, most evaluations based on multimodal measurements with pupillometry predominantly focus on pupil size (52, 53). To fully exploit the relatively high temporal resolution of pupil measurements, future research should aim to enhance the utility of indices based on pupil dynamics, as discussed in Section 3. This will help establish pupil measurements as a more integral tool complementary to multimodal neuroimaging.
5 Conclusion
This perspective highlighted recent advances in the study of neuroimaging and behavioral data, with a focus on pupillometry. The targeted spatiotemporal scales vary depending on the modalities used, as also on the segregation and interactions of different levels of neural activity. Future research is expected to place greater emphasis on interactions within deep brain regions that involve a broad range of fast temporal scales, which can be captured by pupillometry but not fully by EEG or fMRI. Consequently, the adoption of multiple and multimodal measurements and analysis techniques is anticipated to increase, to enhance our understanding of complex neural processes.
Data availability statement
The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.
Author contributions
SN: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing. AS: Writing – original draft, Writing – review & editing. TT: Writing – original draft, Writing – review & editing. ST: Writing – original draft, Writing – review & editing.
Funding
The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This work was supported by JSPS KAKENHI for Grant-in-Aid for Scientific Research [A; grant number JP22H00492 (ST), C; grant number JP22K12183 (SN), grant number JP23K03024 (AS), grant number JP23K06983 (TT), and grant number JP23K07022 (ST)] and Grant-in-Aid for Transformative Research Areas [A; JP20H05921 (SN)] and the Joint Usage/Research Program of the Medical Institute of Developmental Disabilities Research, Showa University (ST).
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
1. Sporns O. The complex brain: connectivity, dynamics, information. Trends Cogn Sci. (2022) 26:1066–7. doi: 10.1016/j.tics.2022.08.002
2. Seguin C, Sporns O, Zalesky A. Brain network communication: concepts, models and applications. Nat Rev Neurosci. (2023) 24:557–74. doi: 10.1038/s41583-023-00718-5
3. Beckmann CF, DeLuca M, Devlin JT, Smith SM. Investigations into resting-state connectivity using independent component analysis. Philos Trans Royal Soc B Biol Sci. (2005) 360:1001–13. doi: 10.1098/rstb.2005.1634
4. Fox MD, Snyder AZ, Vincent JL, Corbetta M, Van Essen DC, Raichle ME. The human brain is intrinsically organized into dynamic, anticorrelated functional networks. Proc Natl Acad Sci USA. (2005) 102:9673–8. doi: 10.1073/pnas.0504136102
5. Damoiseaux JS, Rombouts SA, Barkhof F, Scheltens P, Stam CJ, Smith SM, et al. Consistent resting-state networks across healthy subjects. Proc Natl Acad Sci USA. (2006) 103:13848–53. doi: 10.1073/pnas.0601417103
6. Allen EA, Damaraju E, Plis SM, Erhardt EB, Eichele T, Calhoun VD. Tracking whole-brain connectivity dynamics in the resting state. Cerebr Cortex. (2014) 24:663–76. doi: 10.1093/cercor/bhs352
7. Calhoun VD, Miller R, Pearlson G, Adalı T. The chronnectome: time-varying connectivity networks as the next frontier in fMRI data discovery. Neuron. (2014) 84:262–74. doi: 10.1016/j.neuron.2014.10.015
8. Zalesky A, Fornito A, Cocchi L, Gollo LL, Breakspear M. Time-resolved resting-state brain networks. Proc Natl Acad Sci USA. (2014) 111:10341–6. doi: 10.1073/pnas.1400181111
9. Preti MG, Bolton TA, Van De Ville D. The dynamic functional connectome: state-of-the-art and perspectives. Neuroimage. (2017) 160:41–54. doi: 10.1016/j.neuroimage.2016.12.061
10. Krohn S, von Schwanenflug N, Waschke L, Romanello A, Gell M, Garrett DD, et al. A spatiotemporal complexity architecture of human brain activity. Sci Adv. (2023) 9:eabq3851. doi: 10.1126/sciadv.abq3851
11. Sporns O, Faskowitz J, Teixeira AS, Cutts SA, Betzel RF. Dynamic expression of brain functional systems disclosed by fine-scale analysis of edge time series. Netw Neurosci. (2021) 5:405–33. doi: 10.1162/netn_a_00182
12. Santos FA, Tewarie PK, Baudot P, Luchicchi A, Barros de Souza D, Girier G, et al. Emergence of high-order functional hubs in the human brain. bioRxiv. (2023) 2023:2023–02. doi: 10.1101/2023.02.10.528083
13. Betzel RF, Cutts SA, Tanner J, Greenwell SA, Varley T, Faskowitz J, et al. Hierarchical organization of spontaneous co-fluctuations in densely sampled individuals using fMRI. Netw Neurosci. (2023) 7:926–49. doi: 10.1162/netn_a_00321
14. Thiebaut de Schotten M, Forkel SJ. The emergent properties of the connected brain. Science. 2022) 378:505–10. doi: 10.1126/science.abq2591
15. Logothetis NK. What we can do and what we cannot do with fMRI. Nature. (2008) 453:869–78. doi: 10.1038/nature06976
16. Beppi C, Violante IR, Scott G, Sandrone S. EEG, MEG and neuromodulatory approaches to explore cognition: Current status and future directions. Brain Cogn. (2021) 148:105677. doi: 10.1016/j.bandc.2020.105677
17. Iivanainen J, Mäkinen AJ, Zetter R, Stenroos M, Ilmoniemi RJ, Parkkonen L. Spatial sampling of MEG and EEG based on generalized spatial-frequency analysis and optimal design. NeuroImage. (2021) 245:118747. doi: 10.1016/j.neuroimage.2021.118747
18. Niso G, Krol LR, Combrisson E, Dubarry AS, Elliott MA, François C, et al. Good scientific practice in EEG and MEG research: progress and perspectives. NeuroImage. (2022) 257:119056. doi: 10.1016/j.neuroimage.2022.119056
19. Michel CM, Koenig T. EEG microstates as a tool for studying the temporal dynamics of whole-brain neuronal networks: a review. Neuroimage. (2018) 180:577–93. doi: 10.1016/j.neuroimage.2017.11.062
20. Tewarie P, Liuzzi L, O'Neill GC, Quinn AJ, Griffa A, Woolrich MW, et al. Tracking dynamic brain networks using high temporal resolution MEG measures of functional connectivity. NeuroImage. (2019) 200:38–50. doi: 10.1016/j.neuroimage.2019.06.006
21. Tarailis P, Koenig T, Michel CM, Griškova-Bulanova I. The functional aspects of resting EEG microstates: a systematic review. Brain Topogr. (2023) 9:1–37. doi: 10.1007/s10548-023-00958-9
22. Li Y, Gao J, Yang Y, Zhuang Y, Kang Q, Li X, et al. Temporal and spatial variability of dynamic microstate brain network in disorders of consciousness. CNS Neurosci Therapeut. (2024) 30:e14641. doi: 10.1111/cns.14641
23. Nunez PL, Srinivasan R, Westdorp AF, Wijesinghe RS, Tucker DM, Silberstein RB, et al. EEG coherency: I: statistics, reference electrode, volume conduction, Laplacians, cortical imaging, and interpretation at multiple scales. Electroencephal Clin Neurophysiol. (1997) 103:499–515.
24. Nolte G, Holroyd T, Carver F, Coppola R, Hallett M. “Localizing brain interactions from rhythmic EEG/MEG data,” in The 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Vol. 1. San Francisco, CA: IEEE (2004). p. 998–1001.
25. Eguiluz VM, Chialvo DR, Cecchi GA, Baliki M, Apkarian AV. Scale-free brain functional networks. Phys Rev Lett. (2005) 94:e018102. doi: 10.1103/PhysRevLett.94.018102
26. van den Heuvel M, Mandl R, Luigjes J, Pol HH. Microstructural organization of the cingulum tract and the level of default mode functional connectivity. J Neurosci. (2008) 28:10844–51. doi: 10.1523/JNEUROSCI.2964-08.2008
27. Vinck M, Oostenveld R, Van Wingerden M, Battaglia F, Pennartz CM. An improved index of phase-synchronization for electrophysiological data in the presence of volume-conduction, noise and sample-size bias. Neuroimage. (2011) 55:1548–65. doi: 10.1016/j.neuroimage.2011.01.055
28. van den Heuvel MP, Sporns O. Network hubs in the human brain. Trends Cogn Sci. (2013) 17:683–96. doi: 10.1016/j.tics.2013.09.012
29. Pendl SL, Salzwedel AP, Goldman BD, Barrett LF, Lin W, Gilmore JH, et al. Emergence of a hierarchical brain during infancy reflected by stepwise functional connectivity. Hum Brain Map. (2017) 38:2666–82. doi: 10.1002/hbm.23552
30. Engels MM, Stam CJ, van der Flier WM, Scheltens P, de Waal H, van Straaten EC. Declining functional connectivity and changing hub locations in Alzheimer's disease: an EEG study. BMC Neurology. (2015) 15:145. doi: 10.1186/s12883-015-0400-7
31. Takahashi T, Yamanishi T, Nobukawa S, Kasakawa S, Yoshimura Y, Hiraishi H, et al. Band-specific atypical functional connectivity pattern in childhood autism spectrum disorder. Clin Neurophysiol. (2017) 128:1457–65. doi: 10.1016/j.clinph.2017.05.010
32. Hasegawa C, Takahashi T, Ikeda T, Yoshimura Y, Hiraishi H, Nobukawa S, et al. Effects of familiarity on child brain networks when listening to a storybook reading: a magneto-encephalographic study. NeuroImage. (2021) 241:118389. doi: 10.1016/j.neuroimage.2021.118389
33. Tobe M, Nobukawa S. Functional connectivity estimation by phase synchronization and information flow approaches in coupled chaotic dynamical systems. IEICE Trans Fundament Electr Commun Comput Sci. (2022) 2022:2021EAP1169. doi: 10.1587/transfun.2021EAP1169
34. Farahani FV, Karwowski W, Lighthall NR. Application of graph theory for identifying connectivity patterns in human brain networks: a systematic review. Front Neurosci. (2019) 13:439505. doi: 10.3389/fnins.2019.00585
35. Ismail LE, Karwowski W. A graph theory-based modeling of functional brain connectivity based on EEG: a systematic review in the context of neuroergonomics. IEEE Access. (2020) 8:155103–35. doi: 10.1109/ACCESS.2020.3018995
36. Pegg EJ, Taylor JR, Keller SS, Mohanraj R. Interictal structural and functional connectivity in idiopathic generalized epilepsy: a systematic review of graph theoretical studies. Epilep Behav. (2020) 106:107013. doi: 10.1016/j.yebeh.2020.107013
37. Lujan MR, Perez-Pozuelo I, Grandner MA. Past, present, and future of multisensory wearable technology to monitor sleep and circadian rhythms. Front Digit Health (2021) 3:721919. doi: 10.3389/fdgth.2021.721919
38. Mahanama B, Jayawardana Y, Rengarajan S, Jayawardena G, Chukoskie L, Snider J, et al. Eye movement and pupil measures: a review. Front Comput Sci. (2022) 3:733531. doi: 10.3389/fcomp.2021.733531
39. Gullett N, Zajkowska Z, Walsh A, Harper R, Mondelli V. Heart rate variability (HRV) as a way to understand associations between the autonomic nervous system (ANS) and affective states: a critical review of the literature. Int J Psychophysiol. (2023) 8:1. doi: 10.1016/j.ijpsycho.2023.08.001
40. Iwama S, Takemi M, Eguchi R, Hirose R, Morishige M, Ushiba J. Two common issues in synchronized multimodal recordings with EEG: Jitter and Latency. Neurosci Res. (2023) 30:518625. doi: 10.1101/2022.11.30.518625
41. Foster RG. Sleep, circadian rhythms and health. Interface Focus. (2020) 10:20190098. doi: 10.1098/rsfs.2019.0098
42. Hyun U, Sohn JW. Autonomic control of energy balance and glucose homeostasis. Exp Mol Med. (2022) 54:370–6. doi: 10.1038/s12276-021-00705-9
43. Aston-Jones G, Cohen JD. An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. Annu Rev Neurosci. (2005) 28:403–50. doi: 10.1146/annurev.neuro.28.061604.135709
44. Viglione A, Mazziotti R, Pizzorusso T. From pupil to the brain: new insights for studying cortical plasticity through pupillometry. Front Neural Circ. (2023) 17:1151847. doi: 10.3389/fncir.2023.1151847
45. Montefusco-Siegmund R, Schwalm M, Jubal ER, Devia C, Egaña JI, Maldonado PE. Alpha EEG activity and pupil diameter coupling during inactive wakefulness in humans. eNeuro. (2022) 9:60. doi: 10.1523/ENEURO.0060-21.2022
46. Lloyd B, de Voogd LD, Mäki-Marttunen V, Nieuwenhuis S. Pupil size reflects activation of subcortical ascending arousal system nuclei during rest. ELife. (2023) 12:e84822. doi: 10.7554/eLife.84822
47. Takahashi K, Sobczak F, Pais-Roldán P, Yu X. Characterizing brain stage-dependent pupil dynamics based on lateral hypothalamic activity. Cerebr Cortex. (2023) 33:10736–49. doi: 10.1093/cercor/bhad309
48. Cazettes F, Reato D, Morais JP, Renart A, Mainen ZF. Phasic activation of dorsal raphe serotonergic neurons increases pupil size. Curr Biol. (2021) 31:192–7. doi: 10.1016/j.cub.2020.09.090
49. Nobukawa S, Shirama A, Takahashi T, Takeda T, Ohta H, Kikuchi M, et al. Identification of attention-deficit hyperactivity disorder based on the complexity and symmetricity of pupil diameter. Sci Rep. (2021) 11:1–14. doi: 10.1038/s41598-021-88191-x
50. Usui S, Stark L. A model for nonlinear stochastic behavior of the pupil. Biol Cybernet. (1982) 45:13–21.
51. Nobukawa S, Shirama A, Takahashi T, Takeda T, Ohta H, Kikuchi M, et al. Pupillometric complexity and symmetricity follow inverted-U curves against baseline diameter due to crossed locus coeruleus projections to the Edinger-Westphal Nucleus. Front Physiol. (2021) 12:92. doi: 10.3389/fphys.2021.614479
52. DiNuzzo M, Mascali D, Moraschi M, Bussu G, Maugeri L, Mangini F, et al. Brain networks underlying eye's pupil dynamics. Front Neurosci. (2019) 13:965. doi: 10.3389/fnins.2019.00965
53. Groot JM, Boayue NM, Csifcsák G, Boekel W, Huster R, Forstmann BU, et al. Probing the neural signature of mind wandering with simultaneous fMRI-EEG and pupillometry. NeuroImage. (2021) 224:117412. doi: 10.1016/j.neuroimage.2020.117412
54. Piu P, Serchi V, Rosini F, Rufa A. A cross-recurrence analysis of the pupil size fluctuations in steady scotopic conditions. Front Neurosci. (2019) 13:407. doi: 10.3389/fnins.2019.00407
55. Sviridova N, Artoni P, Fagiolini M, Hensch TK, Aihara K. “Dynamical characteristics of wild-type mouse spontaneous pupillary fluctuations,” in 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). Mexico: IEEE (2021). p. 853–6.
56. Kumano H, Nobukawa S, Shirama A, Takahashi T, Takeda T, Ohta H, et al. Asymmetric complexity in a pupil control model with laterally imbalanced neural activity in the locus coeruleus: a potential biomarker for attention-deficit/hyperactivity disorder. Neural Comput. (2022) 34:2388–407. doi: 10.1162/neco_a_01545
57. Stam CJ, Nolte G, Daffertshofer A. Phase lag index: assessment of functional connectivity from multi channel EEG and MEG with diminished bias from common sources. Hum Brain Map. (2007) 28:1178–93. doi: 10.1002/hbm.20346
58. Ando M, Nobukawa S, Kikuchi M, Takahashi T. Alteration of neural network activity with aging focusing on temporal complexity and functional connectivity within electroencephalography. Front Aging Neurosci. (2022) 14:793298. doi: 10.3389/fnagi.2022.793298
59. Nobukawa S, Yamanishi T, Ueno K, Mizukami K, Nishimura H, Takahashi T. High phase synchronization in alpha band activity in older subjects with high creativity. Front Hum Neurosci. (2020) 14:420. doi: 10.3389/fnhum.2020.583049
60. Takahashi T, Goto T, Nobukawa S, Tanaka Y, Kikuchi M, Higashima M, et al. Abnormal functional connectivity of high-frequency rhythms in drug-naïve schizophrenia. Clin Neurophysiol. (2018) 129:222–31. doi: 10.1016/j.clinph.2017.11.004
61. Nobukawa S, Yamanishi T, Kasakawa S, Nishimura H, Kikuchi M, Takahashi T. Classification methods based on complexity and synchronization of electroencephalography signals in Alzheimer's disease. Front Psychiat. (2020) 11:255. doi: 10.3389/fpsyt.2020.00255
63. Fair DA, Dosenbach NU, Church JA, Cohen AL, Brahmbhatt S, Miezin FM, et al. Development of distinct control networks through segregation and integration. Proc Natl Acad Sci USA. (2007) 104:13507–12. doi: 10.1073/pnas.0705843104
64. Sporns O, Zwi JD. The small world of the cerebral cortex. Neuroinformatics. (2004) 2:145–62. doi: 10.1385/NI:2:2:145
65. Freeman L. Centrality in social networks conceptual clarification. Soc Netw. 1:215. doi: 10.1016/0378-8733(78)90021-7
66. Tobe M, Nobukawa S, Mizukami K, Kawaguchi M, Higashima M, Tanaka Y, et al. Hub structure in functional network of EEG signals supporting high cognitive functions in older individuals. Front Aging Neurosci. (2023) 15:1130428. doi: 10.3389/fnagi.2023.1130428
67. Takahashi T, Murata T, Omori M, Kosaka H, Takahashi K, Yonekura Y, et al. Quantitative evaluation of age-related white matter microstructural changes on MRI by multifractal analysis. J Neurol Sci. (2004) 225:33–7. doi: 10.1016/j.jns.2004.06.016
68. Ishibashi T, Nobukawa S, Tobe M, Kikuchi M, Takahashi T. Alterations in the hub structure of whole-brain functional networks in patients with drug-naïve schizophrenia: insights from electroencephalography-based research. Psychiat Clin Neurosci Rep. (2024) 3:e164. doi: 10.1002/pcn5.164
69. Sporns O, Honey CJ, Kötter R. Identification and classification of hubs in brain networks. PLoS ONE. (2007) 2:e1049. doi: 10.1371/journal.pone.0001049
70. Misic B, Vakorin VA, Paus T, McIntosh AR. Functional embedding predicts the variability of neural activity. Front Syst Neurosci. (2011) 5:90. doi: 10.3389/fnsys.2011.00090
71. Iinuma Y, Nobukawa S, Mizukami K, Kawaguchi M, Higashima M, Tanaka Y, et al. Enhanced temporal complexity of EEG signals in older individuals with high cognitive functions. Front Neurosci. (2022) 16:878495. doi: 10.3389/fnins.2022.878495
72. Hutchison RM, Womelsdorf T, Allen EA, Bandettini PA, Calhoun VD, Corbetta M, et al. Dynamic functional connectivity: promise, issues, and interpretations. Neuroimage. (2013) 80:360–78. doi: 10.1016/j.neuroimage.2013.05.079
73. Gonzalez-Castillo J, Bandettini PA. Task-based dynamic functional connectivity: recent findings and open questions. Neuroimage. (2018) 180:526–33. doi: 10.1016/j.neuroimage.2017.08.006
74. Song H, Rosenberg MD. Predicting attention across time and contexts with functional brain connectivity. Curr Opin Behav Sci. (2021) 40:33–44. doi: 10.1016/j.cobeha.2020.12.007
75. Jalilianhasanpour R, Ryan D, Agarwal S, Beheshtian E, Gujar SK, Pillai JJ, et al. Dynamic brain connectivity in resting state functional MR imaging. Neuroimag Clin. (2021) 31:81–92. doi: 10.1016/j.nic.2020.09.004
76. Cattarinussi G, Di Giorgio A, Moretti F, Bondi E, Sambataro F. Dynamic functional connectivity in schizophrenia and bipolar disorder: a review of the evidence and associations with psychopathological features. Progr Neuro-Psychopharmacol Biol Psychiat. (2023) 2023:110827. doi: 10.1016/j.pnpbp.2023.110827
77. Guan K, Zhang Z, Chai X, Tian Z, Liu T, Niu H. EEG based dynamic functional connectivity analysis in mental workload tasks with different types of information. IEEE Trans Neural Syst Rehabil Eng. (2022) 30:632–42. doi: 10.1109/TNSRE.2022.3156546
78. Nobukawa S, Kikuchi M, Takahashi T. Changes in functional connectivity dynamics with aging: a dynamical phase synchronization approach. Neuroimage. (2019) 188:357–68. doi: 10.1016/j.neuroimage.2018.12.008
79. Nobukawa S, Ikeda T, Kikuchi M, Takahashi T. Atypical instantaneous spatio-temporal patterns of neural dynamics in Alzheimer's disease. Sci Rep. (2024) 14:88. doi: 10.1038/s41598-023-50265-3
80. Garrett DD, Samanez-Larkin GR, MacDonald SW, Lindenberger U, McIntosh AR, Grady CL. Moment-to-moment brain signal variability: a next frontier in human brain mapping? Neurosci Biobehav Rev. (2013) 37:610–24. doi: 10.1016/j.neubiorev.2013.02.015
81. Sharon O, Fahoum F, Nir Y. Transcutaneous vagus nerve stimulation in humans induces pupil dilation and attenuates alpha oscillations. J Neurosci. (2021) 41:320–30. doi: 10.1523/JNEUROSCI.1361-20.2020
82. Liu Y, Rodenkirch C, Moskowitz N, Schriver B, Wang Q. Dynamic lateralization of pupil dilation evoked by locus coeruleus activation results from sympathetic, not parasympathetic, contributions. Cell Rep. (2017) 20:3099–112. doi: 10.1016/j.celrep.2017.08.094
83. Sobczak F, Pais-Roldán P, Takahashi K, Yu X. Decoding the brain state-dependent relationship between pupil dynamics and resting state fMRI signal fluctuation. Elife. (2021) 10:e68980. doi: 10.7554/eLife.68980
84. Kret ME, Sjak-Shie EE. Preprocessing pupil size data: guidelines and code. Behav Res Methods. (2019) 51:1336–42. doi: 10.3758/s13428-018-1075-y
85. Fink L, Simola J, Tavano A, Lange E, Wallot S, Laeng B. From pre-processing to advanced dynamic modeling of pupil data. Behav Res Methods. (2024) 56:1376–412. doi: 10.3758/s13428-023-02098-1
86. Shirama A, Takeda T, Ohta H, Iwanami A, Toda S, Kato N. Atypical alert state control in adult patients with ADHD: a pupillometry study. PLoS ONE. (2020) 15:e0244662. doi: 10.1371/journal.pone.0244662
87. Johansson B, Balkenius C. A computational model of pupil dilation. Connect Sci. (2018) 30:5–19. doi: 10.1080/09540091.2016.1271401
88. Poynter WD. Pupil-size asymmetry is a physiologic trait related to gender, attentional function, and personality. Lateral Asymmetr Body Brain Cogn. (2017) 22:654–70. doi: 10.1080/1357650X.2016.1268147
89. Reimer J, Froudarakis E, Cadwell CR, Yatsenko D, Denfield GH, Tolias AS. Pupil fluctuations track fast switching of cortical states during quiet wakefulness. Neuron. (2014) 84:355–62. doi: 10.1016/j.neuron.2014.09.033
90. Reimer J, McGinley MJ, Liu Y, Rodenkirch C, Wang Q, McCormick DA, et al. Pupil fluctuations track rapid changes in adrenergic and cholinergic activity in cortex. Nat Commun. (2016) 7:1–7. doi: 10.1038/ncomms13289
91. Raut RV, Snyder AZ, Mitra A, Yellin D, Fujii N, Malach R, et al. Global waves synchronize the brain's functional systems with fluctuating arousal. Sci Adv. (2021) 7:eabf2709. doi: 10.1126/sciadv.abf2709
92. van der Wel P, van Steenbergen H. Pupil dilation as an index of effort in cognitive control tasks: a review. Psychon Bullet Rev. (2018) 25:2005–15. doi: 10.3758/s13423-018-1432-y
93. Murphy PR, Vandekerckhove J, Nieuwenhuis S. Pupil-linked arousal determines variability in perceptual decision making. PLoS Comput Biol. (2014) 10:e1003854. doi: 10.1371/journal.pcbi.1003854
94. Nakamura NH, Fukunaga M, Oku Y. Respiratory fluctuations in pupil diameter are not maintained during cognitive tasks. Respirat Physiol Neurobiol. (2019) 265:68–75. doi: 10.1016/j.resp.2018.07.005
95. Schriver BJ, Perkins SM, Sajda P, Wang Q. Interplay between components of pupil-linked phasic arousal and its role in driving behavioral choice in Go/No-Go perceptual decision-making. Psychophysiology. (2020) 57:e13565. doi: 10.1111/psyp.13565
96. Gotham KO, Siegle GJ, Han GT, Tomarken AJ, Crist RN, Simon DM, et al. Pupil response to social-emotional material is associated with rumination and depressive symptoms in adults with autism spectrum disorder. PLoS ONE. (2018) 13:e0200340. doi: 10.1371/journal.pone.0200340
97. Artoni P, Piffer A, Vinci V, LeBlanc J, Nelson CA, Hensch TK, et al. Deep learning of spontaneous arousal fluctuations detects early cholinergic defects across neurodevelopmental mouse models and patients. Proc Natl Acad Sci USA. (2019) 2019:201820847. doi: 10.1073/pnas.1820847116
98. Bast N, Banaschewski T, Dziobek I, Brandeis D, Poustka L, Freitag CM. Pupil dilation progression modulates aberrant social cognition in autism spectrum disorder. Aut Res. (2019) 2019:aur.2178. doi: 10.1002/aur.2178
Keywords: cognitive function, complexity, emergence, graph analysis, functional connectivity, neuroimaging, multimodal data, pupillometry
Citation: Nobukawa S, Shirama A, Takahashi T and Toda S (2024) Recent trends in multiple metrics and multimodal analysis for neural activity and pupillometry. Front. Neurol. 15:1489822. doi: 10.3389/fneur.2024.1489822
Received: 02 September 2024; Accepted: 13 November 2024;
Published: 02 December 2024.
Edited by:
Qi Wang, Columbia University, United StatesReviewed by:
Kuk-In Jang, PANAXTOS Corporate Research Institute, Republic of KoreaCharles Rodenkirch, Cornell Tech, United States
Copyright © 2024 Nobukawa, Shirama, Takahashi and Toda. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Sou Nobukawa, bm9idWthd2EmI3gwMDA0MDtjcy5pdC1jaGliYS5hYy5qcA==