Meng-Yun Wang
Helge J. Zöllner
Meryem A. Yücel
Karsten Specht- 1Department of Biological and Medical Psychology, University of Bergen, Bergen, Norway
- 2Mohn Medical Imaging and Visualization Centre (MMIV), Department of Radiology, Haukeland University Hospital, Bergen, Norway
- 3The Russell H. Morgan Department of Radiology and Radiological Science, Johns Hopkins University School of Medicine, Baltimore, MD, United States
- 4F.M. Kirby Research Center for Functional Brain Imaging, Kennedy Krieger Institute, Baltimore, MD, United States
- 5Neurophotonics Center, Department of Biomedical Engineering, Boston University, Boston, MA, United States
- 6Department of Education, UiT The Arctic University of Norway, Tromsø, Norway
Editorial on the Research Topic
Variability and reproducibility of brain imaging
The human brain processes a constant stream of sensory information, necessitating advanced filtering and cognitive processing for tasks like recognizing dynamic facial expressions (Wang and Yuan, 2021). Advances in noninvasive neuroimaging methods, such as electroencephalogram (EEG) (Wang et al., 2020a), magnetic resonance imaging (MRI) (Lu et al., 2021), and near-infrared spectroscopy (NIRS) (Wang et al., 2020b) have deepened our understanding of how the human brain works over the last decades (Finn et al., 2023). However, the reliability and reproducibility of the results have been recently questioned and exposed in the spotlight (Cognitive neuroscience at the crossroads, 2022), highlighting the need to address and clarify the integrity of the research field (Revisiting doubt in neuroimaging research, 2022).
Hence, the main goal of this Research Topic is to address and explore which factors influence the reliability and reproducibility of brain imaging results and provide practical perspectives and insights to enhance them. We have collected four articles for the Research Topic, whose contributions were discussed as follows.
The first article explored the contributing factors to the brain age index. Brain age is the age gap between chronological age and predicated age from brain imaging data (Cole and Franke, 2017). There is evidence showing that brain age is associated with common brain disorders (Kaufmann et al., 2019), and it has been considered as a potential biomarker for various psychiatric disorders (Cole and Franke, 2017). However, the contribution to brain age from non-brain imaging factors besides neuroimaging data has not been explored. Korbmacher et al. have explored this issue and proposed a Bio-psycho-social model for predicting brain age. The study utilized the UK Biobank data (Miller et al., 2016) and compared different model configurations. They discovered that the Bio-psycho-social model can partially explain the brain age variance which is comparable to the contribution of the diffusion MRI approach. Additionally, they found large variability in gender and ethnicity differences in brain age. They then suggest that in future studies, the effects of ethnicity, cognitive factors, gender, as well as health and lifestyle factors should be controlled to observe the bio-psycho-social factor impacts on brain age.
The second article touched upon the reliability of the resting-state (rs-)fMRI. Put simply, rs-fMRI captures systematic, non-random variations in brain activation in the absence of a specific task. These activations are indirectly reflected in regional fluctuations in the level of oxygenation of the blood (known as BOLD) (Ogawa et al., 1992), which has been widely used to explore the functional brain activation (Lu et al., 2020) and brain networks (Damoiseaux et al., 2006; Bullmore and Sporns, 2009; van den Heuvel and Pol, 2010). There is evidence suggesting that the time-of-day could be associated with reduced global signal fluctuation and functional connectivity (Orban et al., 2020) alongside the time-of-year effect (Wang et al., 2023; Zhang et al., 2023). However, it's still unclear how the time-of-day effect, along with other factors such as age and gender, could sway the results of longitudinal neuroimaging data. Vaisvilaite et al. addressed this issue by leveraging the BETULA dataset, a longitudinal study on aging (Nilsson et al., 1997), and employing the cross-spectral dynamic causal modeling (DCM) (Friston et al., 2003), which puts the temporal fluctuations of resting-state BOLD signal into the focus. Specifically, they explored eight DCM models and found that the time-of-day effect together with gender significantly influences the parameters defining the BOLD signal. Therefore, they suggest that when collecting longitudinal neuroimaging datasets, time of day should be considered a covariate in addition to age and gender.
The third article discussed the reliability of the magnetic resonance spectroscopy (MRS) measurement in multi-site/vendor studies. MRS allows for non-invasive investigation of neurometabolic profiles in health and disease, yielding metabolite estimates from a localized volume (Oz et al., 2014; Wilson et al., 2019). Widely used in clinical settings (Oz et al., 2014), MRS has recently been applied to identify IDH-mutated gliomas (Branzoli and Marjanska, 2020) and explore the association between behavior and MRS-derived neurotransmitters (Li et al., 2022). It is suggested that the MRS-derived metabolite estimates are reliable within the participants (Wang et al., 2024). In large-scale clinical studies, a multi-site/vendor design is often favored to achieve the necessary statistical power. However, the impact of different sites and scanners on metabolite estimates is frequently overlooked (Považan et al., 2020). La et al. investigated how to account for these differences in a pediatric concussion dataset comprising 545 short-TE MRS measurements, collected across six MRI scanners with two MRI vendors at five scanning sites. They used four general linear models and three linear mixed-effects models to investigate site- and vendor effects on the quantitative estimates of five major neuro-metabolites. They found that different analysis strategies of controlling site, vendor, and scanner in MRS data generated different results, from which they advocate that the ComBat harmonization for clinical MRS data (Bell et al., 2022) should be utilized to remove the site and vendor effects.
The fourth article is even more clinically related, exploring the repeatability of 2D and 4D flow MRI measurement of intracranial blood flow and pulsatility. Arterial stiffening impacts blood vessel compliance and regulation and is considered a contributing factor to various neurological conditions (Poels et al., 2012). To noninvasively examine arterial stiffening specifically the blood flow velocity, 2D phase-contrast MRI is dominantly used (McCauley et al., 1995). However, it can only measure a few vessels per scan. On the contrary, the 4D flow MRI can evaluate multiple vessels simultaneously, making it an attractive alternative (Terada et al., 2022). Yet, the repeatability, reliability, and consistency of 4D flow compared to 2D across intracranial vessels remain uncertain. In the study, Morgan et al. aimed to answer this scientific inquiry. They used various statistical methods including intra-rater reliability, inter-method conformity, and test-retest repeatability to assess the repeatability of the pulsatility index and mean flow among patients with small vessel disease and healthy controls. They found that the 4D flow MRI possesses repeatable and reliable PI measurements and the mean blood flow measurement. Therefore, they have suggested the 4D flow MRI can be confidently used to assess the pulsatility across the major cerebral vessels.
We appreciate the contributions of the authors of these four articles, which hopefully could stimulate and elicit further discussions about the topic: the variability and reliability of brain imaging. Indeed, we have witnessed collective initiatives to enhance the robustness of the neuroimaging results (Korbmacher et al., 2023), such as best practices in the data analysis (Nichols et al., 2017; Boudewyn et al., 2023), guidelines for reporting or publishing (Poldrack et al., 2008; Keil et al., 2014; Lin et al., 2021; Yücel et al., 2021), and expert consensus (Choi and Kreis, 2021). We firmly believe by acknowledging the problems we have in neuroimaging studies and adopting the best practices, the neuroimaging results will be more robust and reliable.
Author contributions
M-YW: Writing—original draft, Writing—review & editing, Conceptualization. HZ: Writing—review & editing, Conceptualization. MY: Writing—review & editing, Conceptualization. KS: Writing—review & editing, Conceptualization.
Funding
The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The author(s) declared that they were an editorial board member of Frontiers, at the time of submission. This had no impact on the peer review process and the final decision.
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
Bell, T. K., Godfrey, K. J., Ware, A. L., Yeates, K. O., and Harris, A. D. (2022). Harmonization of multi-site MRS data with ComBat. Neuroimage 257:119330. doi: 10.1016/j.neuroimage.2022.119330
Boudewyn, M. A., Erickson, M. A., Winsler, K., Ragland, J. D., Yonelinas, A., Frank, M., et al. (2023). Managing EEG studies: how to prepare and what to do once data collection has begun. Psychophysiology 60:e14365. doi: 10.1111/psyp.14365
Branzoli, F., and Marjanska, M. (2020). Magnetic resonance spectroscopy of isocitrate dehydrogenase mutated gliomas: current knowledge on the neurochemical profile. Curr. Opin. Neurol. 33, 413–421. doi: 10.1097/WCO.0000000000000833
Bullmore, E. T., and Sporns, O. (2009). Complex brain networks: graph theoretical analysis of structural and functional systems. Nat. Rev. Neurosci. 10, 186–198. doi: 10.1038/nrn2575
Choi, I. Y., and Kreis, R. (2021). Advanced methodology for in vivo magnetic resonance spectroscopy. NMR Biomed. 34:e4504. doi: 10.1002/nbm.4504
Cole, J. H., and Franke, K. (2017). Predicting age using neuroimaging: innovative brain ageing biomarkers. Trends Neurosci. 40, 681–690. doi: 10.1016/j.tins.2017.10.001
Damoiseaux, J. S., Rombouts, S. A., Barkhof, F., Scheltens, P., Stam, C. J., Smith, S. M., et al. (2006). Consistent resting-state networks across healthy subjects. Proc. Natl Acad. Sci. USA. 103, 13848–13853. doi: 10.1073/pnas.0601417103
Finn, E. S., Poldrack, R. A., and Shine, J. M. (2023). Functional neuroimaging as a catalyst for integrated neuroscience. Nature 623, 263–273. doi: 10.1038/s41586-023-06670-9
Friston, K. J., Harrison, L., and Penny, W. (2003). Dynamic causal modelling. Neuroimage 19, 1273–1302. doi: 10.1016/S1053-8119(03)00202-7
Kaufmann, T., van der Meer, D., Doan, N. T., Schwarz, E., Lund, M. J., Agartz, I., et al. (2019). Common brain disorders are associated with heritable patterns of apparent aging of the brain. Nat. Neurosci. 22, 1617–1623. doi: 10.1038/s41593-019-0471-7
Keil, A., Debener, S., Gratton, G., Junghöfer, M., Kappenman, E. S., Luck, S. J., et al. (2014). Committee report: publication guidelines and recommendations for studies using electroencephalography and magnetoencephalography. Psychophysiology 51, 1–21. doi: 10.1111/psyp.12147
Korbmacher, M., Azevedo, F., Pennington, C. R., Hartmann, H., Pownall, M., Schmidt, K., et al. (2023). The replication crisis has led to positive structural, procedural, and community changes. Commun. Psychol. 1:3. doi: 10.1038/s44271-023-00003-2
Li, H., Heise, K.-F., Chalavi, S., Puts, N. A. J., Edden, R. A. E., Swinnen, S. P., et al. (2022). The role of MRS-assessed GABA in human behavioral performance. Prog. Neurobiol. 212:102247. doi: 10.1016/j.pneurobio.2022.102247
Lin, A., Andronesi, O., Bogner, W., Choi, I.-Y., Coello, E., Cudalbu, C., et al. (2021). Minimum reporting standards for in vivo magnetic resonance spectroscopy (MRSinMRS): experts' consensus recommendations. NMR Biomed. 34:e4484. doi: 10.1002/nbm.4484
Lu, F., Liu, P., Chen, H., Wang, M., Xu, S., Yuan, Z., et al. (2020). More than just statics: abnormal dynamic amplitude of low-frequency fluctuation in adolescent patients with pure conduct disorder. J. Psychiatr. Res. 131, 60–68. doi: 10.1016/j.jpsychires.2020.08.027
Lu, F., Wang, M., Xu, S., Chen, H., Yuan, Z., Luo, L., et al. (2021). Decreased interhemispheric resting-state functional connectivity in male adolescents with conduct disorder. Brain Imaging Behav. 15, 1201–1210. doi: 10.1007/s11682-020-00320-8
McCauley, T. R., Peña, C. S., Holland, C. K., Price, T. B., and Gore, J. C. (1995). Validation of volume flow measurements with cine phase-contrast MR imaging for peripheral arterial waveforms. J. Magn. Reson. Imaging 5, 663–668. doi: 10.1002/jmri.1880050608
Miller, K. L., Alfaro-Almagro, F., Bangerter, N. K., Thomas, D. L., Yacoub, E., Xu, J., et al. (2016). Multimodal population brain imaging in the UK Biobank prospective epidemiological study. Nat. Neurosci. 19, 1523–1536. doi: 10.1038/nn.4393
Nichols, T. E., Das, S., Eickhoff, S. B., Evans, A. C., Glatard, T., Hanke, M., et al. (2017). Best practices in data analysis and sharing in neuroimaging using MRI. Nat. Neurosci. 20, 299–303. doi: 10.1038/nn.4500
Nilsson, L. G., Adolfsson, R., Bäckman, L., de Frias, C., Molander, B., Nyberg, L., et al. (1997). The Betula prospective cohort study: memory, health and aging. Aging Neuropsychol. Cogn. 4, 1–32. doi: 10.1080/13825589708256633
Ogawa, S., Tank, D. W., Menon, R., Ellermann, J. M., Kim, S. G., Merkle, H., et al. (1992). Intrinsic signal changes accompanying sensory stimulation - functional brain mapping with magnetic-resonance-imaging. Proc. Natl. Acad. Sci. USA. 89, 5951–5955. doi: 10.1073/pnas.89.13.5951
Orban, C., Kong, R., Li, J., Chee, M. W. L., and Yeo, B. T. T. (2020). Time of day is associated with paradoxical reductions in global signal fluctuation and functional connectivity. Plos Biol. 18:e3000602. doi: 10.1371/journal.pbio.3000602
Oz, G., Alger, J. R., Barker, P. B., Bartha, R., Bizzi, A., Boesch, C., et al. (2014). Clinical proton MR spectroscopy in central nervous system disorders. Radiology 270, 658–679. doi: 10.1148/radiol.13130531
Poels, M. M., Zaccai, K., Verwoert, G. C., Vernooij, M. W., Hofman, A., van der Lugt, A., et al. (2012). Arterial stiffness and cerebral small vessel disease the rotterdam scan study. Stroke 43, 2637–2642. doi: 10.1161/STROKEAHA.112.676528
Poldrack, R. A., Fletcher, P. C., Henson, R. N., Worsley, K. J., Brett, M., Nichols, T. E., et al. (2008). Guidelines for reporting an fMRI study. Neuroimage 40, 409–414. doi: 10.1016/j.neuroimage.2007.11.048
Považan, M., Mikkelsen, M., Berrington, A., Bhattacharyya, P. K., Brix, M. K., Buur, P. F., et al. (2020). Comparison of multivendor single-voxel MR spectroscopy data acquired in healthy brain at 26 sites. Radiology 295, 171–180. doi: 10.1148/radiol.2020191037
Revisiting doubt in neuroimaging research (2022). Nat. Neurosci. 25, 833–834. doi: 10.1038/s41593-022-01125-2
Terada, M., Takehara, Y., Isoda, H., Wakayama, T., and Nozaki, A. (2022). Technical background for 4D flow MR imaging. Magn. Reson. Med. Sci. 21, 267–277. doi: 10.2463/mrms.rev.2021-0104
van den Heuvel, M. P., and Pol, H. E. H. (2010). Exploring the brain network: a review on resting-state fMRI functional connectivity. Eur. Neuropsychopharmacol. 20, 519–534. doi: 10.1016/j.euroneuro.2010.03.008
Wang, M.-Y., Yuan, A., Zhang, J., Xiang, Y., and Yuan, Z. (2020b). Functional near-infrared spectroscopy can detect low-frequency hemodynamic oscillations in the prefrontal cortex during steady-state visual evoked potential-inducing periodic facial expression stimuli presentation. Vis. Comput. Ind. Biomed. Art 3:28. doi: 10.1186/s42492-020-00065-7
Wang, M.-Y., Zhang, Z.-M., Miao, X., Xiaohong, X., and Yuan, Z. (2020a). Electrophysiological evidence of attentional avoidance in sub-clinical individuals with obsessive-compulsive symptoms. IEEE Access 8, 91020–91027. doi: 10.1109/ACCESS.2020.2994452
Wang, M. Y., Korbmacher, M., Eikeland, R., Craven, A. R., and Specht, K. (2024). The intra-individual reliability of H-MRS measurement in the anterior cingulate cortex across 1 year. Hum. Brain Mapp. 45:e26531. doi: 10.1002/hbm.26531
Wang, M. Y., Korbmacher, M., Eikeland, R., and Specht, K. (2023). “The long night effect on the brain functional organization, in The Organization for Human Brain Mapping 2023 Annual Conference (Montreal).
Wang, M. Y., and Yuan, Z. (2021). EEG decoding of dynamic facial expressions of emotion: evidence from SSVEP and causal cortical network dynamics. Neuroscience 459, 50–58. doi: 10.1016/j.neuroscience.2021.01.040
Wilson, M., Andronesi, O., Barker, P. B., Bartha, R., Bizzi, A., Bolan, P. J., et al. (2019). Methodological consensus on clinical proton MRS of the brain: review and recommendations. Magn. Reson. Med. 82, 527–550. doi: 10.1002/mrm.27742
Yücel, M. A., Lühmann, A. V., Scholkmann, F., Gervain, J., Dan, I., Ayaz, H., et al. (2021). Best practices for fNIRS publications. Neurophotonics 8:012101. doi: 10.1117/1.NPh.8.1.012101
Keywords: neuroimaging, reliability, fMRI, MRS, variability, replication crisis
Citation: Wang M-Y, Zöllner HJ, Yücel MA and Specht K (2024) Editorial: Variability and reproducibility of brain imaging. Front. Psychol. 15:1386948. doi: 10.3389/fpsyg.2024.1386948
Received: 16 February 2024; Accepted: 04 March 2024;
Published: 13 March 2024.
Edited and reviewed by: Sara Palermo, University of Turin, Italy
Copyright © 2024 Wang, Zöllner, Yücel and Specht. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Meng-Yun Wang, mengyun.wang@uib.no