Methodological challenges of measuring brain volumes and cortical thickness in idiopathic normal pressure hydrocephalus with a surface-based approach

Identifying disease-specific imaging features of idiopathic Normal Pressure Hydrocephalus (iNPH) is crucial to develop accurate diagnoses, although the abnormal brain anatomy of patients with iNPH creates challenges in neuroimaging analysis. We quantified cortical thickness and volume using FreeSurfer 7.3.2 in 19 patients with iNPH, 28 patients with Alzheimer's disease (AD), and 30 healthy controls (HC). We noted the frequent need for manual correction of the automated segmentation in iNPH and examined the effect of correction on the results. We identified statistically significant higher proportion of volume changes associated with manual edits in individuals with iNPH compared to both HC and patients with AD. Changes in cortical thickness and volume related to manual correction were also partly correlated with the severity of radiological features of iNPH. We highlight the challenges posed by the abnormal anatomy in iNPH when conducting neuroimaging analysis and emphasise the importance of quality checking and correction in this clinical population.

Introduction

Idiopathic Normal Pressure Hydrocephalus (iNPH) is a neurological condition that affects approximately 0.3–3%, of individuals aged 60 and above (Jaraj et al., 2014). It is characterized by alterations in cerebrospinal fluid dynamics, leading to the enlargement of the ventricles to maintain a stable intracranial pressure (Carswell, 2022). A triad of symptoms; gait apraxia, urinary incontinence, and cognitive deficits, result from this compensatory ventricular expansion, which stretches and distorts the surrounding parenchyma (Carswell, 2022). Therapeutic redirection of cerebrospinal fluid to an area of lower pressure (i.e., shunting) can dramatically improve symptoms (Carswell, 2022).

iNPH occurs in the elderly population in which traditional neurodegenerative diseases are common (Jaraj et al., 2014), and identifying iNPH-specific clinical and imaging features is paramount to being able to distinguish these disorders. The anatomical features of iNPH introduce methodological challenges in neuroimaging analysis. Reduced callosal angle, ventriculomegaly, and disproportionately enlarged subarachnoid space hydrocephalus (DESH) are some of such distinctive features of iNPH seen on brain imaging (Hashimoto et al., 2010). Here, we would like to address potential limitations associated with the use of FreeSurfer,¹ a software used for the analysis and visualization of brain imaging data, in this specific patient group.

One notable advantage of FreeSurfer is its ability to employ a fully automated pipeline, enabling the segmentation of the brain into regions of interest. It is freely available, widely used and there is extensive experience within the field in implementing it within analysis pipelines aiding reproducibility. FreeSurfer registers the volume with the MNI305 atlas. It performs a surface-based reconstruction of the cortex, which classifies voxels as either white or non-white matter based on voxel intensity and neighbour constraints, and a volume-based stream for volume labelling of each point (voxel) of the brain mask (Dale et al., 1999; Fischl et al., 2002). It derives the white matter surface as the interface between the white and gray matter, and the pial surface as the boundary between the pial and cerebrospinal fluid (CSF). Cortical thickness and volumes can then be quantified in 34 different regions derived from the Desikan-Killiany atlas. This automated process is considerably less laborious and less prone to bias than manual regions of interest segmentation.

Quality control and manual editing can be performed to rectify errors related to skull stripping, grey-white matter segmentation, and intensity normalization.² Several studies have compared the outputs of the FreeSurfer’s pipeline with and without manual edits in groups of healthy adults, individuals with genetic disorders, and severe head injuries and found mixed results (McCarthy et al., 2015; Guenette et al., 2018; Waters et al., 2019). There is also limited research investigating the significance of the manual editing step in clinical populations with extremely abnormal brain morphology, which can impact the registration and segmentation analysis stages.

Methods

We evaluated the importance of manually correcting the segmentation output produced by FreeSurfer 7.3.2³ on the MRI scans of 19 patients with iNPH, 28 patients with Alzheimer’s disease, and 30 healthy controls (HC). To improve the readability of the results and reduce multiple comparisons, the 34 regions segmented by FreeSurfer where clustered to derive cortical thickness and volumes for the frontal, temporal, parietal, occipital and cingulate lobes (see Footnote 1). Between-group differences in age and gender were analysed using Kruskal-Wallis test and Chi-Square test, respectively. All scans were visually checked to ensure their quality met appropriate research standards. We then ran the FreeSurfer recon-all command using the -bigventricles flag. Of the 19 iNPH patients, 12 were classified as probable, 4 as possible and 3 as asymptomatic iNPH, as defined by international criteria (Relkin et al., 2005). Among the 16 symptomatic iNPH patients, 15 received a lumbar puncture and had their CSF samples analysed to determine the presence of comorbid AD pathology. Amyloid deposition was detected in two patients. Radiological features of iNPH were assessed and calculated by a neuroradiologist. Participants were scanned on a 3 T Siemens scanner as part of a wider ongoing study run by the UK Dementia Research Institute, Care Research & Technology Centre focused on using sensor technology to monitor behaviours of people living with dementia. We visually inspected each output and performed manual editing when necessary (Figure 1). Wilcoxon signed-ranked tests were used to compare the FreeSurfer’s measurements (volumes and cortical thickness) before and after manual edits while accounting for non-normally distributed data. Between-group differences in these changes were assessed via repeated measures ANOVA, followed by two-tailed t-tests with FDR correction for post-hoc comparisons. Finally, exploratory Spearman correlations were conducted between changes in cortical thickness/volumes pre and post manual correction and radiological features of iNPH (i.e., Radscale score, Evan’s index, callosal angle and DESH score). To assess the potential for rectifying FreeSurfer’s inaccuracies through alternative pre-processing software, we conducted two additional evaluations. First, we integrated the HD-BET tool for skull stripping before executing the FreeSurfer recon-all command. Notably, HD-BET has exhibited superior performance compared to various widely used brain extraction algorithms, even in the presence of brain pathology (Isensee et al., 2019). Additionally, we experimented with running the FreeSurfer recon-all command using a combination of T1 and FLAIR scans.⁴

Figure 1

Figure 1. Left: Output of the Freesurfer’s recon-all command before the manual editing step for one subject. DESH features (i.e., enlarged ventricles, widened sylvian fissure and tight high convexity) are marked in red. Right: Output of the manual editing step for the same subject showing reduced cortical thickness.

This study was approved by the Health Research Authority’s London-Surrey Borders Research Ethics Committee (19/LO/0102) and the Health Research Authority’s London-Central Research Ethics Committee (18/LO/0249). All participants gave written and/or electronic consent.

Results

HC (14 females, mean age = 75.58 years, SD = 6.07), AD patients (12 females, mean age 75.25 years, SD =7.64 years) and NPH patients (7 females, mean age = 71.58 years, SD = 5.92 years) did not differ significantly in terms of gender. No significant age difference was found between HC and AD patients. Conversely, iNPH participants were significantly younger than AD and HC (p = 0.01). The iNPH patients had a mean Evan’s Index of 0.38 (SD = 0.04), mean callosal angle of 75.7 (SD = 15.83), mean Radscale score of 9.3 (SD = 1.51) and mean DESH score of 7.06 (SD = 1.77). Out of the 19 scans of patients with iNPH, 3 failed the segmentation step (Figure 2) and 15 required extensive manual corrections (Figure 1). Of the 3 patients whose Freesurfer segmentation failed, 2 were asymptomatic. Of the 28 patients with Alzheimer’s disease, one failed the segmentation and 4 required manual corrections. In the HC group, only 2 participants needed manual editing of the segmentation output. No corrections of the white matter surface were required in any study group. In the iNPH group, manual edits aimed to improve the removal of skull and rectify inaccuracies in defining the pial surface, which had extended into the dura and skull. Following manual correction, the parietal, frontal and temporal regions exhibited the most substantial differences; with volume and cortical thickness measures decreasing bilaterally across the group (Table 1). Wilcoxon signed-ranked tests comparing these measurements before and after manual edits did not reach significance, although we may have been underpowered by small participants’ number. Repeated measures ANOVA indicated an effect of group on the delta values of the volumes (F_(19,630) = 2.84, p < 0.001), but not cortical thickness (p > 0.05), which suggest potential higher reliability of this measure relative to volumes. Between group differences were observed for the delta values of the frontal, parietal, temporal and cingulate volumes (Table 1). In Supplementary Table S1, we also report the differences in cortical thickness and volumes before and after manual correction for all the 34 individual regions segmented by FreeSurfer and the between-group comparisons of the delta values.

Figure 2

Figure 2. Example of failed segmentation for one iNPH patient. Due to the presence of oedema, the pial and white matter surface are wrongly estimated around the ventricles and extend into the CSF space.

Table 1

Table 1. Values of cortical thickness and volumes before and after manual correction for the 5 main lobes (left and right), and between-group comparisons of the delta values (FDR corrected).

Spearman correlations showed that, in the left and right temporal lobe, cortical thickness changes significantly correlated with the Radscale score (rho = 0.61/60, p = 0.01), and the left temporal lobe volume also correlated with the callosal angle in isolation (rho = −0.53,p = 0.03). We also found a significant correlation between DESH scores and the change in volume of the right occipital lobe (rho = −0.51, p = 0.04).

The additional evaluations of FreeSurfer’s accuracy using HD BET in the pre-processing step revealed 7 segmentation failures and the necessity of multiple manual edits in 11 scans. Similarly, employing a combination of FLAIR and T1 images also led to seven segmentation failures and required manual corrections in five outputs.

Discussion

The higher proportion of scans requiring correction within our sample of iNPH patients relative to the AD and HC groups underlines the importance of conducting and reporting this quality check in this group – which is not consistently done (Cogswell et al., 2021).

Whilst we acknowledge that the overall effect of the correction in these data is minor, it is important to note that this is a relatively small sample size and that we employed a conservative manual correction approach to mitigate the risk of bias associated with human judgement; the effect of this correction process might become substantial enough to influence results significantly when conducting larger studies. Our findings also reveal a statistically higher proportion of volume alterations attributed to manual edits in individuals with iNPH compared to both healthy controls and patients with AD. Changes in cortical thickness were in part correlated with the severity of radiological features of iNPH and underline the importance of exercising caution when using FreeSurfer with severe hydrocephalus. It is important to underline that one significant limitation of this study is the subjectivity of the visual inspections and manual corrections, which are prone to human error. However, we have followed the methodology and guidelines provided by the developers to mitigate bias and maximise consistency in our approach.⁵

The challenge for the field lies in establishing brain biomarkers that can differentiate between iNPH and other dementia types with overlapping clinical presentations and radiological features, such as ventriculomegaly, in order to identify patients to target with therapeutic shunting. Previous studies have demonstrated abnormal cortical thickening in the parietal lobe, and in the high convexity of the frontal, parietal, and occipital lobes in iNPH patients compared to healthy individuals and patients with Alzheimer’s disease (Moore et al., 2012; Kang et al., 2020; Bianco et al., 2022). Studies have suggested that cortical thickening may be characteristic of iNPH and related to the ventricular expansion, which leads to compression and stretching of the brain tissue, which may then reduce the cerebrospinal fluid space in the high convexity regions (Kang et al., 2020; Han et al., 2022). We cautiously suggest that increased cortical thickness and tightness of the high-convexity space increase the likelihood of FreeSurfer failing to delineate the pia from the dura and hence erroneously classifying extra voxels to grey matter. If not corrected, these inaccuracies may provide even further and exaggerate evidence of increased cortical thickness and volumes in these areas. Interestingly, segmentation errors did not affect the white matter surface. FreeSurfer’s failures seems to specifically impact the delineation of the pial surface. Since this is measured as the interface between the pial and the CSF, these inaccuracies could arise from the reduced CSF space and the tight high-convexity regions resulting from ventricular expansion.

In light of the challenges discussed above, we propose that researchers consider the likely lengthy process of manual correction that is required when using FreeSurfer in this clinical group and encourage the reporting of the completion of this step so that readers can have confidence in any associated results. However, there is a need for further, large-scale iNPH studies to reliably identify disease-specific biomarkers. In this case, conducting laborious manual corrections which can take several hours per subject (Lotan et al., 2022) may be unfeasible and introduce the likelihood of bias, especially given the challenges in blinding raters to the clinical group each scan comes from, given such apparent structural abnormalities.

With this in mind, alternative automatised software and analysis techniques with superior accuracy have been developed and may be preferential (Carass et al., 2017; Shao et al., 2019; Billot et al., 2023). Nevertheless, as shown above, FreeSurfer is still being widely used in current studies. This may be due to some limitations of these alternative tools. These in fact do not always provide segmentation of the individual compartments of the ventricles or are validated in small subsamples of iNPH patients (Shiee et al., 2011; Roy et al., 2015; Shao et al., 2019), do not improve the required processing time relative to FreeSurfer (Ellingsen et al., 2016), are not always freely available (Shao et al., 2019) or easily accessible as FreeSurfer (Ellingsen et al., 2016), or need manual delineation of new atlases when employed with new scanners (Roy et al., 2015).

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors upon reasonable request.

Ethics statement

The studies involving humans were approved by the Health Research Authority’s London-Surrey Borders Research Ethics Committee (19/LO/0102) and the Health Research Authority’s London-Central Research Ethics Committee (18/LO/0249). All participants gave written and/or electronic consent. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

MG: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Writing – original draft, Writing – review & editing, Visualization. MD: Data curation, Visualization, Writing – review & editing, Resources. MK: Data curation, Writing – review & editing, Resources. AG: Data curation, Writing – review & editing, Investigation, Resources. TP: Supervision, Writing – review & editing, Methodology. AH: Supervision, Writing – review & editing. DS: Supervision, Writing – review & editing, Funding acquisition, Resources. PM: Conceptualization, Investigation, Supervision, Writing – review & editing. CC: Conceptualization, Investigation, Methodology, Project administration, Resources, Supervision, Writing – original draft.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This work was supported by UK Dementia Research Institute and the Alzheimer’s society DRI-CORE2020-CRT. DS reports grants from NIHR Invention for innovation (i4i) (II-LB-0715-20006). CC reports grants from Medical Research Council (PA1804).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The handling editor VC declared a past co-authorship with the author DS.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnins.2024.1366029/full#supplementary-material

Footnotes

1. ^http://surfer.nmr.mgh.harvard.edu/

2. ^https://surfer.nmr.mgh.harvard.edu/fswiki/FsTutorial/TroubleshootingData

3. ^https://surfer.nmr.mgh.harvard.edu/fswiki/CorticalParcellation

4. ^https://surfer.nmr.mgh.harvard.edu/fswiki/recon-all#UsingT2orFLAIRdatatoimprovepialsurfaces

5. ^https://surfer.nmr.mgh.harvard.edu/fswiki/FsTutorial/PialEditsV6.0

References

Bianco, M. G., Quattrone, A., Sarica, A., Vescio, B., Buonocore, J., Vaccaro, M. G., et al. (2022). Cortical atrophy distinguishes idiopathic normal-pressure hydrocephalus from progressive supranuclear palsy: a machine learning approach. Parkinsonism Relat. Disord. 103, 7–14. doi: 10.1016/j.parkreldis.2022.08.007

PubMed Abstract | Crossref Full Text | Google Scholar

Billot, B., Greve, D. N., Puonti, O., Thielscher, A., Van Leemput, K., Fischl, B., et al. (2023). SynthSeg: segmentation of brain MRI scans of any contrast and resolution without retraining. Med. Image Anal. 86:102789. doi: 10.1016/j.media.2023.102789

PubMed Abstract | Crossref Full Text | Google Scholar

Carass, A., Shao, M., Li, X., Dewey, B. E., Blitz, A. M., Roy, S., et al. (2017). Whole brain Parcellation with pathology: validation on Ventriculomegaly patients. Patch Based Tech. Med. Imaging, 10530:20–28. doi: 10.1007/978-3-319-67434-6_3

PubMed Abstract | Crossref Full Text | Google Scholar

Carswell, C. (2022). Idiopathic normal pressure hydrocephalus: historical context and a contemporary guide. Pract. Neurol. 23, 15–22. doi: 10.1136/pn-2021-003291

Crossref Full Text | Google Scholar

Cogswell, P. M., Weigand, S. D., Wiste, H. J., Gunter, J. L., Graff-Radford, J., Jones, D. T., et al. (2021). CSF dynamics as a predictor of cognitive progression. NeuroImage 232:117899. doi: 10.1016/j.neuroimage.2021.117899

PubMed Abstract | Crossref Full Text | Google Scholar

Dale, A. M., Fischl, B., and Sereno, M. I. (1999). Cortical surface-based analysis. NeuroImage 9, 179–194. doi: 10.1006/nimg.1998.0395

Crossref Full Text | Google Scholar

Ellingsen, L. M., Roy, S., Carass, A., Blitz, A. M., Pham, D. L., and Prince, J. L. (2016). “Segmentation and labeling of the ventricular system in normal pressure hydrocephalus using patch-based tissue classification and multi-atlas labeling” Proc. SPIE Int. Soc. Opt. Eng. 9784:97840G. doi: 10.1117/12.2216511

Crossref Full Text | Google Scholar

Fischl, B., Salat, D. H., Busa, E., Albert, M., Dieterich, M., Haselgrove, C., et al. (2002). Whole brain segmentation. Neuron 33, 341–355. doi: 10.1016/S0896-6273(02)00569-X

Crossref Full Text | Google Scholar

Guenette, J. P., Stern, R. A., Tripodis, Y., Chua, A. S., Schultz, V., Sydnor, V. J., et al. (2018). Automated versus manual segmentation of brain region volumes in former football players. Neuroimage Clin. 18, 888–896. doi: 10.1016/j.nicl.2018.03.026

PubMed Abstract | Crossref Full Text | Google Scholar

Han, J., Kim, M. N., Lee, H. W., Jeong, S. Y., Lee, S. W., Yoon, U., et al. (2022). Distinct volumetric features of cerebrospinal fluid distribution in idiopathic normal-pressure hydrocephalus and Alzheimer’s disease. Fluids Barriers CNS 19:66. doi: 10.1186/s12987-022-00362-8

PubMed Abstract | Crossref Full Text | Google Scholar

Hashimoto, M., Ishikawa, M., Mori, E., and Kuwana, N. (2010). Diagnosis of idiopathic normal pressure hydrocephalus is supported by MRI-based scheme: a prospective cohort study. Cerebrospinal Fluid Res. 7:18. doi: 10.1186/1743-8454-7-18

PubMed Abstract | Crossref Full Text | Google Scholar

Isensee, F., Schell, M., Pflueger, I., Brugnara, G., Bonekamp, D., Neuberger, U., et al. (2019). Automated brain extraction of multisequence MRI using artificial neural networks. Hum. Brain Mapp. 40, 4952–4964. doi: 10.1002/hbm.24750

PubMed Abstract | Crossref Full Text | Google Scholar

Jaraj, D., Rabiei, K., Marlow, T., Jensen, C., Skoog, I., and Wikkelso, C. (2014). Prevalence of idiopathic normal-pressure hydrocephalus. Neurology 82, 1449–1454. doi: 10.1212/WNL.0000000000000342

PubMed Abstract | Crossref Full Text | Google Scholar

Kang, K., Han, J., Lee, S. W., Jeong, S. Y., Lim, Y. H., Lee, J. M., et al. (2020). Abnormal cortical thickening and thinning in idiopathic normal-pressure hydrocephalus. Sci. Rep. 10:21213. doi: 10.1038/s41598-020-78067-x

PubMed Abstract | Crossref Full Text | Google Scholar

Lotan, E., Damadian, B. E., Rusinek, H., Griffin, M., Ades-Aron, B., Lu, N., et al. (2022). Quantitative imaging features predict spinal tap response in normal pressure hydrocephalus. Neuroradiology 64, 473–481. doi: 10.1007/s00234-021-02782-z

PubMed Abstract | Crossref Full Text | Google Scholar

McCarthy, C. S., Ramprashad, A., Thompson, C., Botti, J. A., Coman, I. L., and Kates, W. R. (2015). A comparison of FreeSurfer-generated data with and without manual intervention. Front. Neurosci. 9:9. doi: 10.3389/fnins.2015.00379

Crossref Full Text | Google Scholar

Moore, D. W., Kovanlikaya, I., Heier, L. A., Raj, A., Huang, C., Chu, K. W., et al. (2012). A pilot study of quantitative MRI measurements of ventricular volume and cortical atrophy for the differential diagnosis of normal pressure hydrocephalus. Neurol. Res. Int. 2012, 1–6. doi: 10.1155/2012/718150

PubMed Abstract | Crossref Full Text | Google Scholar

Relkin, N., Marmarou, A., Klinge, P., Bergsneider, M., and Black, P. M. (2005). Diagnosing idiopathic Normal-pressure hydrocephalus. Neurosurgery 57, S2-4–S2-16. doi: 10.1227/01.NEU.0000168185.29659.C5

Crossref Full Text | Google Scholar

Roy, S., He, Q., Sweeney, E., Carass, A., Reich, D. S., Prince, J. L., et al. (2015). Subject-specific sparse dictionary learning for atlas-based brain MRI segmentation. IEEE J. Biomed. Health Inform. 19, 1598–1609. doi: 10.1109/JBHI.2015.2439242

PubMed Abstract | Crossref Full Text | Google Scholar

Shao, M., Han, S., Carass, A., Li, X., Blitz, A. M., Shin, J., et al. (2019). Brain ventricle parcellation using a deep neural network: application to patients with ventriculomegaly. Neuroimage Clin. 23:101871. doi: 10.1016/j.nicl.2019.101871

PubMed Abstract | Crossref Full Text | Google Scholar

Shiee, N., Bazin, P. L., Cuzzocreo, J. L., Blitz, A., and Pham, D. L. (2011). Segmentation of brain images using adaptive atlases with application to Ventriculomegaly. Inf. Process. Med. Imaging, 22, 1–12. doi: 10.1007/978-3-642-22092-0_1

PubMed Abstract | Crossref Full Text | Google Scholar

Waters, A. B., Mace, R. A., Sawyer, K. S., and Gansler, D. A. (2019). Identifying errors in FreeSurfer automated skull stripping and the incremental utility of manual intervention. Brain Imaging Behav. 13, 1281–1291. doi: 10.1007/s11682-018-9951-8

PubMed Abstract | Crossref Full Text | Google Scholar