The functional connectivity and neuropsychology underlying mental planning operations: data from the digital clock drawing test

We examined the construct of mental planning by quantifying digital clock drawing digit placement accuracy in command and copy conditions, and by investigating its underlying neuropsychological correlates and functional connectivity. We hypothesized greater digit misplacement would associate with attention, abstract reasoning, and visuospatial function, as well as functional connectivity from a major source of acetylcholine throughout the brain: the basal nucleus of Meynert (BNM). Participants (n = 201) included non-demented older adults who completed all metrics within 24 h of one another. A participant subset met research criteria for mild cognitive impairment (MCI; n = 28) and was compared to non-MCI participants on digit misplacement accuracy and expected functional connectivity differences. Digit misplacement and a comparison dissociate variable of total completion time were acquired for command and copy conditions. a priori fMRI seeds were the bilateral BNM. Command digit misplacement is negatively associated with semantics, visuospatial, visuoconstructional, and reasoning (p’s < 0.01) and negatively associated with connectivity from the BNM to the anterior cingulate cortex (ACC; p = 0.001). Individuals with MCI had more misplacement and less BNM-ACC connectivity (p = 0.007). Total completion time involved posterior and cerebellar associations only. Findings suggest clock drawing digit placement accuracy may be a unique metric of mental planning and provide insight into neurodegenerative disease.

Introduction

Mental planning is characterized by the ability to recruit and effectively deploy selected neurocognitive operations necessary to bring a given task to fruition and is critical for the successful completion of many complex activities of daily living such as managing personal finances, organizing work schedules, arranging travel plans, and preparing meals (Owen, 1997; Cohen and Conway, 2007). Disrupted mental planning is often present in older adults with mild cognitive impairment (MCI) and dementia (Cohen et al., 2008). Mental planning involves cholinergic nervous system activity from the basal forebrain (Sarter and Paolone, 2011; Demeter and Sarter, 2013) and frontal and posterior brain regions (Furey et al., 2008a, b; Bracco et al., 2014). Mental planning requires diverse neurocognitive operations including executive abilities, semantic knowledge, and visuospatial operations (Luria, 1966).

The Clock Drawing Test (CDT; Freedman et al., 1994; Libon et al., 1996) is a general cognitive screening measure requiring mental planning for successful completion. The CDT’s command condition requires participants to “draw the face of a clock, put in all the numbers and set the hands to 10 after 11,” while the copy condition involves copying a pre-drawn model clock. The CDT command requires the coordination of a number of semantic, visuospatial, and executive functions, while the copy condition primarily revolves around executive functions related to appreciating the visuospatial nature of the model (Price et al., 2011). Comparisons between CDT command and copy conditions with hands set for “10 after 11” has been well investigated and show unique patterns for MCI and dementia types (Libon et al., 1996; Price et al., 2011). To date, however, critical aspects of CDT production, such as digit placement accuracy, have strictly relied on visual inspection from trained neuropsychological professionals. The digital version of the CDT (dCDT; Souillard-Mandar et al., 2016) provides a more precise and objective scoring method to quantify digit placement accuracy, broadening the use of the clock drawing as a screening tool for mental planning deficits in a wide variety of clinical settings.

The current project investigated mental planning abilities via digital clock drawing digit placement accuracy in command and copy conditions. The first aim was to validate the variable of digit placement accuracy as a measure of mental planning. We examined expected associations between digit misplacement and neuropsychological tests of attention, abstract reasoning, and visuospatial function; cognitive operations thought to underlie planning abilities (Fuster, 2015). As part of this aim, we also compared digit placement accuracy within a sample of individuals with and without MCI. Reduced visual attention is associated with reduced overall cognition (Okonkwo et al., 2008), and adults with MCI display significant mental planning deficits (Yamamoto et al., 2004). We hypothesized MCI participants would display less dCDT placement accuracy, particularly in the command condition given its increased demand on numerous cognitive functions. Second, we explored the neural bases for dCDT digit placement accuracy as a measure of mental planning abilities. Acetylcholine plays a key role in attention and prefrontal functions (Klinkenberg et al., 2011; Fuster, 2015) and is vital for effective mental planning. For this reason, we further hypothesized digit placement accuracy would be associated with functional connectivity from the BNM; a known origin for acetylcholine with vast innervation throughout the brain (Gratwicke et al., 2013). Specifically, we expected that individuals with less digit placement accuracy (or greater digit misplacement) would show less functional connectivity from the BNM. We further expected BNM functional connectivity would be reduced in MCI relative to non-MCI participants due to the general cholinergic reduction in these individuals (Mesulam et al., 1983; Ruberg et al., 1990; Vogels et al., 1990) and that this compromised functional connectivity from the BNM may lead to the cognitive impairment seen in MCI (Whitehouse et al., 1981; Mesulam et al., 1983; Nordberg, 1993; Grothe et al., 2010; Ferreira-Vieira et al., 2016). Lastly, for each examined aim, we compared the cognitive and functional MRI patterns observed with digit placement accuracy to the patterns observed with a common digital clock metric known to associate with numerous areas of cognition: total completion time (Dion et al., 2020). This comparison allowed us to dissociate the cognitive and functional specificity of digit placement accuracy relative to the speed-based metric of total clock drawing time which is known to associate with numerous cognitive domains (Dion et al., 2020).

Methods

Participants

Participant data were prospectively acquired through a National Institutes of Health (NIH) funded study approved by the University of Florida Institutional Review Board (IRB). All participants provided written informed consent and the study was conducted in accordance with the Declaration of Helsinki. Inclusion criteria: age ≥55, English as the primary language, and intact instrumental activities of daily living (IADLs; Lawton and Brody, 1969) assessed with both the participant and their caregiver. Exclusion criteria: major neurocognitive, neurodegenerative, or psychiatric disorder at baseline per the Diagnostic and Statistical Manual of Mental Disorders—Fifth Edition (DSM-V; APA, 2013), significant medical illness limiting lifespan, documented learning disorder, seizure disorder, or other neurological illness, <6th-grade education, substance abuse history, major cardiac disease, or chronic medical illness thought to induce encephalopathy.

Participants were screened using the Telephone Interview for Cognitive Status (TICS; Welsh et al., 1993) and during an in-person interview with a neuropsychologist and trained research coordinator to assess comorbidities, anxiety, depression, neuropsychological functioning, and digital clock drawing (Davis et al., 2011). The same examiner administered all test items. fMRI acquisition was completed within 24 h of behavioral testing. Trained raters scored, double scored, and double entered all behavioral data. We defined MCI with Jak and colleagues’ comprehensive criteria using age-adjusted normative data (Jak et al., 2009).

Neuropsychological measures of interest

Neuropsychological measures were chosen based on their theoretical involvement in mental planning and accurate placement of numbers within the clock face. We converted raw neuropsychological tests score to z-scores using externally published norms (Lezak et al., 2004). Neuropsychological constructs and associated tests:

Processing speed

Digit Symbol, Wechsler Adult Intelligence Scale—3rd edition (WAIS-III), total correct in 120 s; Stroop Word Reading—Word Reading, total correct in 45 s.

Working memory

Letter Number Sequencing, WAIS-III—total number of correct sequences; Digit Span (Backward), WAIS-III—longest span backward.

Episodic memory

Logical Memory, Wechsler Memory Scale—3rd edition (WMS-III)—total delay recall score; Hopkins Verbal Learning Test—Revised (HVLT-R)—delay total recall correct.

Naming/lexical access

Boston Naming Test (BNT)—total correct out of 60. Animal fluency—total correct words in 60 s.

Visuospatial operations

Matrix Reasoning, Wechsler Abbreviated Scale of Intelligence (WASI)—total correct. Judgment of Line Orientation (JOLO)—total number of correct items. Rey Osterrieth Complex Figure—copy score.

Covariates of interest

Age, and cognitive reserve (Stern, 2009). Cognitive reserve, which refers to individual differences in the brain’s ability to offset pathological attacks by relying on premorbid cognitive abilities to maintain cognitive functioning (Stern, 2009), was operationalized by averaging the following estimates of premorbid intelligence: vocabulary knowledge (WASI, vocabulary subtest total correct), word reading using Wide Range Achievement Test (WRAT total score), and years of education (Lezak et al., 2012).

Clock drawing test

Participants complete a clock drawing to command and copy conditions. The command condition requires participants to “draw the face of a clock, put in all the numbers, and set the hands to 10 after 11.” The copy condition requires the participant to draw a clock underneath a pre-drawn model clock.

We captured clock drawing with digital pen technology from Anoto, Inc., which works as a conventional ballpoint pen and measures pen positioning 75 times/s on associated smart article with a spatial resolution of ±0.002 inches. We scored all clock drawings using Clocksketch software, a program developed at MIT, that classifies each pen stroke with over 84% accuracy (Souillard-Mandar et al., 2016). External raters replayed and deconstructed each drawing ensuring accurate scoring (93%–99% inter-rater reliability).

Digit misplacement

The Clocksketch software generates a best-fit clock face for the participant’s drawing so that it has a specified “center point” for the clock. The program then spatially identifies each digit and surrounds the strokes of each digit with a bounding box at their widest and highest most points. The center of this bounding box is used to determine a digit’s location in reference to the clock face center point. The software centers a geometric protractor of 360 degrees about the best-fit clock face. An angle of placement for each digit is then calculated from the clock face’s center to the center of each digit’s bounding box. This angle of placement is then subtracted from the “ideal” angle the digit should be placed at, resulting in a degree of misplacement based on the digit’s deviation from the ideal placement (Figure 1). Because of this formulation, we will hereby refer to digit placement accuracy as “digit misplacement” to reference the deviation from the ideal.

FIGURE 1

Figure 1. Example of a digital clock drawing, with the clock face of best-fit outlined in orange, the center point indicated by the orange and blue crosshairs, and the digit bounding boxes in tan. Close up of digit 1, which should be centered at 60° (green line) but is slightly misplaced (red line crossing through tan bonding boxes center) at 57.502°. The difference between the red and green values is how misplaced the digit is, indicated as “Digit 1 Ideal Difference”. In this case, Digit 1 Ideal Difference is 60 − 57.502 = 2.498° misplaced.

We converted all 12 digits’ degrees of misplacement to absolute values to display the distance from the ideal without directionality. A single score to capture overall digit misplacement, termed “Total Misplacement” was then calculated by summing all the absolute ideal differences, as shown by the equation below. This means that larger values represent greater digit misplacement (a greater value = less-ideally placed digits).

$Misplacement=\sum _{n=1}^{12}\left(\left|IdealAngelofDigit\left(n\right)-ActualAngel\text{ of }Digit\left(n\right)\right|\right)$

To ensure precision in measuring mental planning, we excluded data from participants who rotated the article before or during the production of their drawing, omitted a clock face circle, drew more or less than 12 digits and/or repeated digits. The orientation of the article is justification for removal to avoid misplacement reflecting both the degree of misplacement and the degree of improper orientation. Since the clock face circle is required for the program to generate a center point on which to base the ideal placement, digit misplacement could only be calculated for those with a clock face. Perseverations or absence of any of the typical 12 digits may represent confounding impairments with different underlying cognitive constructs than digit misplacement and therefore such data were removed from this sample. The above criteria resulted in the removal of two participants.

Total clock drawing time

The Clocksketch software also measures total completion time (TCT). This variable reflects the time taken in seconds to complete all elements of the clock drawing, from the initial pen-article contact until the completion of the last pen stroke. TCT in the command condition positively correlates with traditional neuropsychological measures of processing speed, language, working memory, and declarative memory. In contrast, TCT in the copy condition mainly correlates with processing speed and working memory (Dion et al., 2020). Therefore, we used TCT as a comparison variable with known cognitive correlates to better appreciate the cognitive implications of digit misplacement.

Statistical analyses—digital clock drawing and neuropsychological tests

Statistical analyses were completed in SPSS v.25 with statistical significance set at p < 0.05. Using Spearman correlations, we identified participant demographic covariates. We used separate partial correlations between clock drawing variables (digit misplacement and TCT) and neuropsychological measures controlling for cognitive reserve. We addressed MCI group differences after adjustment for the covariate using analysis of covariance (ANCOVA) and normalized the distribution of clock drawing variables using natural log transformation. We corrected each statistical model for multiple comparisons using a Benjamini-Hochberg correction with FDR set to 0.05. These analyses were then conducted with total completion time for comparison.

Neuroimaging parameters

Brain MRI data were acquired on a 3T Siemens Verio scanner with an 8-channel head coil. We ran T1-weighted and resting-state fMRI sequences for each participant. T1-weighted data were acquired with the following parameters: TR: 2,500 ms; TE: 3.77 ms; 176 sagittal 1 mm³ slices, 1 mm isotropic resolution; 256 × 256 × 176 matrix, 7/8 phase partial Fourier, total acquisition time: 9:22. Resting state fMRI data were acquired with participants’ eyes closed and with the following parameters: TR: 2,000 ms; TE: 30 ms; 36 transverse slices; 3.5 mm³ isotropic voxel size, 225 × 225 × 126 matrix. We used Generalized Autocalibrating Partial Parallel Acquisition (GRAPPA) to reduce imaging time (Griswold et al., 2002); total acquisition time: 7:38.

Resting state fMRI

Preprocessing and quality assurance of functional and structural MRI data were performed using the default pipeline implemented in the CONN Toolbox (19. c¹; Whitfield-Gabrieli and Nieto-Castanon, 2012), which included functional scan realignment, interleaved slice-timing correction, co-registration to T1w, spatial normalization, and smoothing according to a full-width half-maximum (FWHM) isotropic Gaussian kernel filter of 8 mm. T1w images were then tissue-type segmented into gray matter, white matter, and cerebrospinal fluid (CSF). Nonlinear normalization to Montreal Neurological Institute (MNI152) was then performed. All preprocessing used Statistical Parametric Mapping v.12 (SPM12) software². Functional scans were subjected to artifact and motion outlier identification using the Artifact Detection Toolbox according to conservative settings (95th percentile of the normative sample). These settings identified time points as outliers if movement from a preceding image exceeded 0.5 mm or if the global mean signal intensity exceeded three standard deviations. Seven of 201 participants (3.5%) who had less than 5 min of scan time remaining after time points were identified as outliers and were therefore removed from the analyses. Outlier time points were included as regressors along with principal components delineated from anatomical noise regions (10 components for white matter, five components for CSF) and realignment parameters during a denoising step. Finally, a 0.008–0.09Hz band-pass filter was applied to the functional data (Hallquist et al., 2013).

fMRI statistical analysis for BNM connectivity

Functional connectivity was calculated using weighted seed-based connectivity (wSBC) maps. The a priori seed was the bilateral BNM, a major source of cortical acetylcholine (Johnston et al., 1981; Rye et al., 1984). The BNM region of interest (ROI) was based on a stereotaxic probability map of magnocellular cell groups in the basal forebrain, defined in the standard MNI space (Zaborszky et al., 2008; Chiang-shan et al., 2014). The ROI was then imported into the CONN Toolbox, masking the ROI with gray matter to include only signals coming from inside each participant’s gray matter mask. Next, a mean signal time course for the BNM was calculated. wSBC values were then calculated by correlating the ROI signal time course and the time series of every other voxel in the brain. The correlations in time series between the seed and all other voxels were then Fisher’s r-to-z transformed. Using the CONN Toolbox (Whitfield-Gabrieli and Nieto-Castanon, 2012), we ran general linear models correlating our variables of interest (command and copy digit misplacement and total completion time) with BNM connectivity, while controlling for age and cognitive reserve, which were demeaned before being entered into the general linear model. Clusters of significant connectivity were thresholded using a parametric Gaussian Random Field Theory approach (Worsley et al., 1996), which removes voxels with uncorrected p < 0.001 and then keeps only clusters that survived a two-tailed correction for multiple comparisons using an FDR p < 0.05. Significant clusters were localized using the Harvard-Oxford Brain Atlas (Desikan et al., 2006) with cerebellar regions from the Automated Anatomical (AAL) Atlas (Tzourio-Mazoyer et al., 2002). Mean r-to-z transformed correlation values for any significant clusters were exported for everyone for additional analyses.

Results

Participants

As part of a prior investigation (Dion et al., 2020), 206 participants completed the neuropsychological protocol and dCDT (Figure 2). We excluded two participants due to concerns revolving around a learning disorder, and one participant due to concerns about Parkinson’s disease (PD). Two additional participants were excluded due to clock drawings having more or less than 12 digits, thus retaining data for 201 participants. Of our retained sample (n = 201), 173 participants were classified as non-MCI, while 28 met the criteria for MCI (Jak et al., 2009). Groups did not differ on age or comorbidities; see demographic characteristics by MCI status in Table 1. The MCI group completed fewer years of education resulting in lower cognitive reserve. Of the whole sample (n = 201), 163 participants completed neuroimaging, and data from seven participants were removed for poor quality fMRI, for a final sample size with valid neuroimaging data of 156. Of the retained sample with neuroimaging data, 131 were classified as non-MCI, while 25 met the criteria for MCI. Table 2 shows the demographics of the whole sample compared to the neuroimaging subsample.

FIGURE 2

Figure 2. Participant exclusion flow chart.

TABLE 1

Table 1. Participant characteristics by MCI status.

TABLE 2

Table 2. Participant characteristics.

Digit misplacement and neuropsychological test performance (whole sample, n = 201)

Command: Greater digit misplacement was associated with worse performance on the BNT (r = −0.319, p < 0.001), Matrix Reasoning (r = −0.318, p < 0.001), Rey Complex Figure Copy (r = −0.207, p = 0.015), and JOLO (r = −0.255, p = 0.002).

Copy: Greater digit misplacement associated with poorer performance on the Matrix Reasoning (r = −0.269, p < 0.001), JOLO (r = −0.198, p = 0.019), and Rey Complex Figure Copy (r = -0.205, p = 0.015; Table 3).

TABLE 3

Table 3. Partial correlations between digit clock drawing variables (Command and Copy) and neuropsychological tests of interest.

Total completion time and neuropsychological test performance (whole sample, n = 201)

Command: Longer or slower completion time negatively and significantly correlated with lower scores on Digit Symbol (r = −0.398, p < 0.001), Stroop Word Reading (r = −0.237, p = 0.005), Letter Number Sequencing (r = −0.322, p < 0.001), Logical Memory (r = −0.312, p < 0.001), and Digit Span Backward (r = −0.184, p = 0.030).

Copy: Longer or slower completion time negatively and significantly correlated with lower scores on Digit Symbol (r = −0.444, p < 0.001), Stroop Word Reading (r = −0.245, p = 0.004), Letter Number Sequencing (r = −0.200, p = 0.018), and Logical Memory (r = −0.211, p = 0.013; Table 3).

Digit misplacement and MCI group differences (non-MCI n = 173, MCI n = 28)

Command: Controlling for cognitive reserve, there was a significant group difference F _(1,186) = 21.08, p < 0.001, η² = 0.103, such that the MCI group displayed greater digit misplacement than non-MCI peers. On average, individuals with MCI exhibited 43 degrees of greater misplacement than non-MCI peers (MCI Mean = 112.18, SD = 63.08; non-MCI Mean = 68.76, SD = 32.59).

Copy: Controlling for cognitive reserve, there was a significant group difference F_(1,187) = 6.99, p = 0.037, η² = 0.023, such that the MCI group displayed greater digit misplacement than non-MCI peers. On average, individuals with MCI exhibited 19 degrees of greater digit misplacement than non-MCI peers (MCI Mean = 84.34, SD = 37.55; non-MCI Mean = 65.26, SD = 25.76).

Resting state fMRI: seed-based connectivity from bilateral BNM to whole brain (neuroimaging subsample n = 156)

Command Digit Misplacement: After controlling for age and cognitive reserve, significant negative associations were found between BNM connectivity with the anterior cingulate cortex (ACC; FDR-corrected p = 0.0002; Table 4 and Figure 3A) such that greater digit misplacement was observed in association with less BNM-ACC connectivity. Copy Digit Misplacement: Statistically not significant.

FIGURE 3

Figure 3. (A) Negative relationships between command total digit misplacement and basal nucleus of Meynert connectivity with the anterior cingulate cortex. (B) Positive relationships between command total digit misplacement and basal nucleus of Meynert connectivity with cerebellum (anterior lobe, superior posterior lobe, and vermis). (C) Negative relationships between command total completion time and basal nucleus of Meynert connectivity with right supramarginal gyrus and contiguous regions, and positive associations with left occipital pole and the left superior posterior lobe of the cerebellum.

TABLE 4

Table 4. Clusters demonstrating significant associations between BNM connectivity and digital clock variables.

Command Clock Drawing Total Completion Time: there were significant associations involving BNM connectivity such that longer command total completion time was associated with greater BNM connectivity with bilateral anterior and superior posterior lobules and the vermis of the cerebellum (FDR-corrected p < 0.002; see Table 4 and Figure 3B). Copy Clock Drawing Total Completion Time: there were significant associations involving BNM connectivity such that a longer total completion time was associated with less BNM connectivity with right supramarginal gyrus and contiguous regions (FDR-corrected p = 0.0003). Longer time to completion is associated with greater connectivity with the left temporal/occipital region and the left superior posterior lobule of the cerebellum (FDR-corrected p = 0.0128; see Table 4 and Figure 3C).

MCI vs. non-MCI resting state connectivity (non-MCI n = 131, MCI n = 25)

Controlling for age and cognitive reserve, MCI and non-MCI groups significantly differed on BNM-ACC connectivity such that the MCI group (Mean = 0.05, SD = 0.16) showed lower BNM-ACC connectivity than the non-MCI group (Mean = 0.17, SD = 0.17; t₍₁₅₀₎ = 2.35, p = 0.0202). Both groups displayed similar relationships between BNM-ACC connectivity and command digit misplacement (see Figure 4). Wilcoxon rank sum tests showed no group differences in command total time to completion or copy total time to completion with BNM connectivity values (BNM to the five clusters, see Table 4). P values > 0.456.

FIGURE 4

Figure 4. Scatter plot representing the relationships between BNM to ACC functional connectivity and command digit misplacement.

Discussion

Findings suggest clock drawing digit misplacement has expected associations with cognitive metrics and functional connections to the Basal Nucleus of Meynert (BNM). Command digit misplacement is associated with neuropsychological metrics of semantic knowledge, abstract reasoning, visuospatial operations, and visuoconstructional operations with analogous results in the copy condition. These findings extend upon prior research reporting upon neuropsychological constructs of the CDT (Libon et al., 1996; Cosentino et al., 2004; Price et al., 2011; Dion et al., 2020). Further, digit misplacement showed a unique association with BNM-ACC resting state connectivity such that greater digit misplacement was associated with less BNM-ACC connectivity. Research suggests the ACC plays an evaluative role in monitoring behavioral errors and adapting accordingly (Valenstein et al., 1987; Botvinick et al., 2001, 2004). Reduced digit accuracy may therefore represent reduced connectivity from the BNM to the ACC.

The current study also identified MCI and non-MCI group differences such that older adults with MCI showed greater digit misplacement and lower functional connectivity from BNM. The MCI findings extend upon research demonstrating how neurodegenerative conditions such as Alzheimer’s disease (AD) are associated with forebrain function and altered afferent cholinergic projection into cortical and subcortical regions (Kuhl et al., 1996). The MCI and non-MCI group differences are also consistent with prior research showing that, in comparison to healthy controls, older adults with MCI display lower functional connectivity from BNM to the insula/claustrum, involved in the integration of various cortical inputs (Li et al., 2017). ACC input to the claustrum is thought to play a criterial role in modulating visuospatial operations and impulsivity in animal models (Robbins, 2002; Li et al., 2017; White and Mathur, 2018). Further, Cantero and colleagues demonstrated that in amnestic MCI, BNM atrophy is associated with structural changes in their innervated regions (Cantero et al., 2017).

By contrast, the functional networks associated with total completion time suggest a different cognitive-neuronal fMRI profile. First, faster copy clock drawing was associated with increased connectivity from BNM to left and right lateral supramarginal gyri. Research suggests the supramarginal gyrus is involved in a wide variety of tasks including requiring cross-modal cognitive operations (Butters et al., 1970; Geschwind, 1974) as well as tasks assessing visuomotor abilities and mental flexibility of motor action (Hanakawa et al., 2008). Second, for the command, and to a lesser extent the copy condition, slower clock drawing is associated with increased BNM-cerebellar connectivity, particularly in regions IV through VI, including the vermis. Since cerebellar acetylcholine comes primarily from brainstem nuclei afferents (Zhang et al., 2016), increased connectivity with the BNM may indicate abnormal organization, since the cerebellum is involved in motor, cognition, emotion, and vestibular functions (Schmahmann and Sherman, 1998; Manto and Mariën, 2015). Prior work has linked gray matter volume in the posterior lobules to processing speed in multiple sclerosis (Moroso et al., 2017). Task-based functional neuroimaging indicates increased anterior cerebellar activity in MCI (for a summary see Jacobs et al., 2018) and associations with lobular volumes suggests associations with executive function but also visuomotor coordination and memory (Kansal et al., 2017). Lastly, we found no group differences between MCI and non-MCI groups in command total time to completion or copy total time to completion with BNM connectivity values (BNM to the five clusters). These findings suggest these relationships appear to be somewhat independent of MCI status, unlike digit misplacement and BNM-ACC connectivity. We encourage additional research examining subtle behavior features of clock drawing and MCI profiles.

We recognize study limitations. First, the MCI sample was small, limiting statistical power. Thus, the MCI analyses may have been underpowered to detect other meaningful differences across groups. Given this small sample, we chose not to investigate potential differences across MCI phenotypes (i.e., single/multiple-domain amnestic MCI, dysexecutive MCI, mixed MCI). Second, our MCI group completed fewer years of education, likely influencing our measure of cognitive reserve, hence our statistical correction for cognitive reserve. Third, the study has limitations including an obvious lack of ethnic and educational diversity. Given this underrepresentation, we did not investigate differences across ethno-racial or education groups. Future studies need to expand this research to ethno-racially, geospatially, and more educationally diverse samples. Fourth, since the dCDT scoring program fits the best-fit ellipse to the clock face as the first step in deriving digit misplacement, highly asymmetrical clocks may require additional statistical corrections. Additionally, to create a single continuous measure of digit misplacement for the entire drawing, misplacement for all 12 digits was summed. For this reason, we included clock data only from those who drew 12 digits. Our results should, therefore, be interpreted in such context and may not be generalizable to highly impaired clock drawings containing more or less than 12 digits. Future research should include clock drawings of individuals with different forms of cognitive impairment, such as vascular dementia, AD, and surgical samples. Another potential limitation is the size of the BNM region of interest. Although the BNM region of interest (ROI) was based on a stereotaxic probability map of magnocellular cell groups in the basal forebrain, defined in the standard MNI space (Zaborszky et al., 2008; Chiang-shan et al., 2014), we note that the ROI is larger than the BNM proper. Consequently, data acquired are not fully specific to the BNM and include some of the surrounding gray matter. Lastly, we used the variable of total time to completion only for comparison purposes with digit misplacement. Previous work from our team shows that individuals with MCI take significantly longer to draw their clocks to command (i.e., on average 10 s longer) relative to non-MCI peers (Dion et al., 2020). Future studies should investigate the neurobiological underpinnings of total time to completion.

Despite limitations, our investigation has a number of strengths. The study examined two distinct subtle behavioral features captured via digital technology methods. Digital technology cognitive assessment and rapid behavioral capture will be increasingly due to the need for community-based cognitive screening (Diaz-Orueta et al., 2020). The scientific questions were based on theoretical constructs of mental planning and functional connectivity involving the BNM. The study examined an objective metric of digit misplacement (which is in contrast to the longstanding subjective measurement of digit placement accuracy based on visual scoring). This novel approach reduces the potential for scoring error and removes scoring subjectivity, thereby increasing accessibility for clinical use. Further, the metric was compared to a more common dCDT metric of total time to completion, thereby providing us with insight into the unique functional connectivity associations of the digit misplacement metric. Our large sample of non-demented older adults is also well-characterized based on a comprehensive neuropsychological protocol, allowing for high confidence in the correlations between neuropsychological domains and neuroimaging. We additionally classified our MCI sample based on rigorous published criteria validated through neuroimaging and pathology reports (Jak et al., 2009), thus allowing for meaningful results/interpretation in a smaller sample.

Study findings add to the existing literature documenting the advantages of appreciating subtle behavioral nuances and their contribution to our understanding of brain-behavior relationships. The incorporation of digital technology of clock drawing allows for the identification of subtle behaviors, such as digit misplacement, that could otherwise not accurately be categorized via traditional article and pencil testing (Emrani et al., 2020, 2021; Davoudi et al., 2021; Dion et al., 2021). Digit misplacement appears to assess a unique aspect of cognition in older adults and may help identify those with compromised attentional networks specific to the BNM and ACC. The comparison variable of total time to completion associates with multiple cognitive domains including processing speed (Dion et al., 2020) and, in the current study, involved posterior and cerebellar associations. Within rapid screening settings for community interventions (e.g., Amini et al., 2019), these rapid clinical metrics may have a unique advantage. We encourage the investigation of the predictive value of digitally acquired metrics from tests such as clock drawing on brain metrics and/or cognitive status. Data will guide future investigations addressing digitally acquired clock drawing profiles in neurodegenerative disorders.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by University of Florida Institutional Review Board. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

Conceptualization and methodology: CD, JT, EF, DL, and CP. Data acquisition: KR and JT. Data curation: CD, JT, EF, AD, and MA. Writing—original draft preparation: CD and EF. Writing—review and editing: CD, JT, EF, MW, DP, RD, DL, KH, CG, and CP. Supervision: DL and CP. All authors contributed to the article and approved the submitted version.

Funding

This work was supported by the National Institute of Health (grant nos. R01AG055337; P50AG047266; R01NR014181; R01NS082386; UL1R001427; K07AG066813); and the National Science Foundation (grant no. 1404333).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

1. ^ http://web.conn-toolbox.org/home
2. ^ www.fil.ion.ucl.ac.uk/spm/software/spm12/

References

Amini, S., Crowley, S., Hizel, L., Arias, F., Libon, D. J., Tighe, P., et al. (2019). Feasibility and rationale for incorporating frailty and cognitive screening protocols in a preoperative anesthesia clinic. Anesth. Analg. 129:830. doi: 10.1213/ANE.0000000000004190

PubMed Abstract | CrossRef Full Text | Google Scholar

APA (2013). Diagnostic and Statistical Manual of Mental Disorders (DSM-5^®). Washington, DC: American Psychiatric Publishing.

Google Scholar

Botvinick, M. M., Braver, T. S., Barch, D. M., Carter, C. S., and Cohen, J. D. (2001). Conflict monitoring and cognitive control. Psychol. Rev. 108:624. doi: 10.1037/0033-295x.108.3.624

PubMed Abstract | CrossRef Full Text | Google Scholar

Botvinick, M. M., Cohen, J. D., and Carter, C. S. (2004). Conflict monitoring and anterior cingulate cortex: an update. Trends Cogn. Sci. 8, 539–546. doi: 10.1016/j.tics.2004.10.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Bracco, L., Bessi, V., Padiglioni, S., Marini, S., and Pepeu, G. (2014). Do cholinesterase inhibitors act primarily on attention deficit? A naturalistic study in Alzheimer’s disease patients. J. Alzheimers Dis. 40, 737–742. doi: 10.3233/JAD-131154

PubMed Abstract | CrossRef Full Text | Google Scholar

Butters, N., Barton, M., and Brody, B. A. (1970). Role of the right parietal lobe in the mediation of cross-modal associations and reversible operations in space. Cortex 6, 174–190. doi: 10.1016/s0010-9452(70)80026-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Cantero, J. L., Zaborszky, L., and Atienza, M. (2017). Volume loss of the nucleus basalis of Meynert is associated with atrophy of innervated regions in mild cognitive impairment. Cereb. Cortex 27, 3881–3889. doi: 10.1093/cercor/bhw195

PubMed Abstract | CrossRef Full Text | Google Scholar

Chiang-shan, R. L., Ide, J. S., Zhang, S., Hu, S., Chao, H. H., Zaborszky, L., et al. (2014). Resting state functional connectivity of the basal nucleus of meynert in humans: in comparison to the ventral striatum and the effects of age. Neuroimage 97, 321–332. doi: 10.1016/j.neuroimage.2014.04.019

PubMed Abstract | CrossRef Full Text | Google Scholar

Cohen, G., and Conway, M. A. (2007). Memory in the Real World. London: Psychology Press.

Google Scholar

Cohen, R. A., Salloway, S., and Sweet, L. H. (2008). “Neuropsychiatric aspects of disorders of attention,” in The American Psychiatric Publishing Textbook of Neuropsychiatry and Behavioral Neurosciences, eds S. C. Yudofsky, and R. E. Hales (Washington, DC: American Psychiatric Publishing, Inc.), 405–444.

Google Scholar

Cosentino, S., Jefferson, A., Chute, D. L., Kaplan, E., and Libon, D. J. (2004). Clock drawing errors in dementia: neuropsychological and neuroanatomical considerations. Cogn. Behav. Neurol. 17, 74–84. doi: 10.1097/01.wnn.0000119564.08162.46

PubMed Abstract | CrossRef Full Text | Google Scholar

Davis, R., Penney, D., Pittman, D., Libon, D., Swenson, R., Kaplan, E., et al. (2011). The digital clock drawing test (dCDT) I: development of a new computerized quantitative system. Int. Neuropsychol. Soc. 17, 274 doi: 10.1017/S1355617711000415

PubMed Abstract | CrossRef Full Text | Google Scholar

Davoudi, A., Dion, C., Formanski, E., Frank, B. E., Amini, S., Matusz, E. F., et al. (2021). Normative references for graphomotor and latency digital clock drawing metrics for adults age 55 and older: operationalizing the production of a normal appearing clock. J. Alzheimers Dis. 82, 59–70. doi: 10.3233/JAD-201249

PubMed Abstract | CrossRef Full Text | Google Scholar

Demeter, E., and Sarter, M. (2013). Leveraging the cortical cholinergic system to enhance attention. Neuropharmacology 64, 294–304. doi: 10.1016/j.neuropharm.2012.06.060

PubMed Abstract | CrossRef Full Text | Google Scholar

Desikan, R. S., Ségonne, F., Fischl, B., Quinn, B. T., Dickerson, B. C., Blacker, D., et al. (2006). An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest. Neuroimage 31, 968–980. doi: 10.1016/j.neuroimage.2006.01.021

PubMed Abstract | CrossRef Full Text | Google Scholar

Diaz-Orueta, U., Blanco-campal, A., Lamar, M., Libon, D. J., and Burke, T. (2020). Marrying past and present neuropsychology: is the future of the process-based approach technology-based? Front. Psychol. 11:361. doi: 10.3389/fpsyg.2020.00361

PubMed Abstract | CrossRef Full Text | Google Scholar

Dion, C., Arias, F., Amini, S., Davis, R., Penney, D., Libon, D. J., et al. (2020). Cognitive correlates of digital clock drawing metrics in older adults with and without mild cognitive impairment. J. Alzheimers Dis. 75, 73–83. doi: 10.3233/JAD-191089

PubMed Abstract | CrossRef Full Text | Google Scholar

Dion, C., Frank, B. E., Crowley, S. J., Hizel, L. P., Rodriguez, K., Tanner, J., et al. (2021). Parkinson’s disease cognitive phenotypes show unique clock drawing features when measured with digital technology. J. Parkinsons Dis. 11, 779–791. doi: 10.3233/JPD-202399

PubMed Abstract | CrossRef Full Text | Google Scholar

Emrani, S., Baliga, S., Lamar, M., Price, C. C., Au, R., Wasserman, V., et al. (2020). Working memory and mild cognitive impairment: digital intra-response latency, subcortical nuclei and neuropsychological test performance: using digital technology to drive clinical innovation in brain-behavior relationships: a new era in neuropsychology. Alzheimers Dement. 16:e038141. doi: 10.1002/alz.038141

CrossRef Full Text | Google Scholar

Emrani, S., Lamar, M., Price, C. C., Baliga, S., Wasserman, V. J., Matusz, E., et al. (2021). Assessing the capacity for mental manipulation in patients with statically-determined mild cognitive impairment using digital technology. Explor. Med. 2, 86–97. doi: 10.37349/emed.2021.00034

CrossRef Full Text | Google Scholar

Ferreira-Vieira, T. H., Guimaraes, I. M., Silva, F., and Ribeiro, F. (2016). Alzheimers disease: targeting the cholinergic system. Curr. Neuropharmacol. 14, 101–115. doi: 10.2174/1570159x13666150716165726

PubMed Abstract | CrossRef Full Text | Google Scholar

Freedman, M., Leach, L., Kaplan, E., Winocur, G., Shulman, K., Delis, D. C., et al. (1994). Clock Drawing: A Neuropsychological Analysis. USA: Oxford University Press.

Google Scholar

Furey, M. L., Pietrini, P., Haxby, J. V., and Drevets, W. C. (2008a). Selective effects of cholinergic modulation on task performance during selective attention. Neuropsychopharmacology 33, 913–923. doi: 10.1038/sj.npp.1301461

PubMed Abstract | CrossRef Full Text | Google Scholar

Furey, M. L., Ricciardi, E., Schapiro, M. B., Rapoport, S. I., and Pietrini, P. (2008b). Cholinergic enhancement eliminates modulation of neural activity by task difficulty in the prefrontal cortex during working memory. J. Cogn. Neurosci. 20, 1342–1353. doi: 10.1162/jocn.2008.20092

PubMed Abstract | CrossRef Full Text | Google Scholar

Fuster, J. M. (2015). The Prefrontal Cortex. New York: Academic Press.

Google Scholar

Geschwind, N. (1974). Disconnexion syndromes in animals and man. Brain 1, 105–236.

Google Scholar

Gratwicke, J., Kahan, J., Zrinzo, L., Hariz, M., Limousin, P., Foltynie, T., et al. (2013). The nucleus basalis of Meynert: a new target for deep brain stimulation in dementia? Neurosci. Biobehav. Rev. 37, 2676–2688. doi: 10.1016/j.neubiorev.2013.09.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Griswold, M. A., Jakob, P. M., Heidemann, R. M., Nittka, M., Jellus, V., Wang, J., et al. (2002). Generalized autocalibrating partially parallel acquisitions (GRAPPA). Magn. Reson. Med. 47, 1202–1210. doi: 10.1002/mrm.10171

PubMed Abstract | CrossRef Full Text | Google Scholar

Grothe, M., Zaborszky, L., Atienza, M., Gil-Neciga, E., Rodriguez-Romero, R., Teipel, S. J., et al. (2010). Reduction of basal forebrain cholinergic system parallels cognitive impairment in patients at high risk of developing Alzheimer’s disease. Cereb. Cortex 20, 1685–1695. doi: 10.1093/cercor/bhp232

PubMed Abstract | CrossRef Full Text | Google Scholar

Hallquist, M. N., Hwang, K., and Luna, B. (2013). The nuisance of nuisance regression: spectral misspecification in a common approach to resting-state fMRI preprocessing reintroduces noise and obscures functional connectivity. Neuroimage 82, 208–225. doi: 10.1016/j.neuroimage.2013.05.116

PubMed Abstract | CrossRef Full Text | Google Scholar

Hanakawa, T., Dimyan, M. A., and Hallett, M. (2008). Motor planning, imagery and execution in the distributed motor network: a time-course study with functional MRI. Cereb. Cortex 18, 2775–2788. doi: 10.1093/cercor/bhn036

PubMed Abstract | CrossRef Full Text | Google Scholar

Jacobs, H. I., Hopkins, D. A., Mayrhofer, H. C., Bruner, E., Van Leeuwen, F. W., Raaijmakers, W., et al. (2018). The cerebellum in Alzheimer’s disease: evaluating its role in cognitive decline. Brain 141, 37–47. doi: 10.1093/brain/awx194

PubMed Abstract | CrossRef Full Text | Google Scholar

Jak, A., Bondi, M. W., Delano-Wood, L., Wierenga, C., Corey-Bloom, J., Salmon, D. P., et al. (2009). Quantification of five neuropsychological approaches to defining mild cognitive impairment. Am. J. Geriatr. Psychiatry 17, 368–375. doi: 10.1097/JGP.0b013e31819431d5

PubMed Abstract | CrossRef Full Text | Google Scholar

Johnston, M., Mckinney, M., and Coyle, J. (1981). Neocortical cholinergic innervation: a description of extrinsic and intrinsic components in the rat. Exp. Brain Res. 43, 159–172. doi: 10.1007/BF00237760

PubMed Abstract | CrossRef Full Text | Google Scholar

Kansal, K., Yang, Z., Fishman, A. M., Sair, H. I., Ying, S. H., Jedynak, B. M., et al. (2017). Structural cerebellar correlates of cognitive and motor dysfunctions in cerebellar degeneration. Brain 140, 707–720. doi: 10.1093/brain/aww327

PubMed Abstract | CrossRef Full Text | Google Scholar

Klinkenberg, I., Sambeth, A., and Blokland, A. (2011). Acetylcholine and attention. Behav. Brain Res. 221, 430–442. doi: 10.1016/j.bbr.2010.11.033

PubMed Abstract | CrossRef Full Text | Google Scholar

Kuhl, D. E., Minoshima, S., Fessler, J. A., Ficaro, E. P., Wieland, D. M., Koeppe, R. A., et al. (1996). In vivo mapping of cholinergic terminals in normal aging, Alzheimer’s disease and Parkinson’s disease. Ann. Neurology 40, 399–410. doi: 10.1002/ana.410400309

PubMed Abstract | CrossRef Full Text | Google Scholar

Lawton, M. P., and Brody, E. M. (1969). Assessment of older people: self-maintaining and instrumental activities of daily living. The Gerontologist 9, 179–186.

Google Scholar

Lezak, M. D., Howieson, D. B., Loring, D. W., and Fischer, J. S. (2004). Neuropsychological Assessment. USA: Oxford University Press.

Google Scholar

Lezak, M., Howieson, D., and Loring, D. (2012). Neuropsychological Assessment. New York: Oxford University Press.

Google Scholar

Li, H., Jia, X., Qi, Z., Fan, X., Ma, T., Ni, H., et al. (2017). Altered functional connectivity of the basal nucleus of Meynert in mild cognitive impairment: a resting-state fMRI study. Front. Aging Neurosci. 9:127. doi: 10.3389/fnagi.2017.00127

PubMed Abstract | CrossRef Full Text | Google Scholar

Libon, D. J., Malamut, B. L., Swenson, R., Sands, L. P., and Cloud, B. S. (1996). Further analyses of clock drawings among demented and nondemented older subjects. Arch. Clin. Neuropsychol. 11, 193–205.

Google Scholar

Luria, A. R. (1966). “The problem of localization of functions in the cerebral cortex,” in Higher Cortical Functions in Man (Boston, MA: Springer), 3–36.

Google Scholar

Manto, M., and Mariën, P. (2015). Schmahmann’s syndrome-identification of the third cornerstone of clinical ataxiology. Cerebellum Ataxias 2, 1–5. doi: 10.1186/s40673-015-0023-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Mesulam, M. M., Mufson, E. J., Levey, A. I., and Wainer, B. H. (1983). Cholinergic innervation of cortex by the basal forebrain: cytochemistry and cortical connections of the septal area, diagonal band nuclei, nucleus basalis (substantia innominata) and hypothalamus in the rhesus monkey. J. Comp. Neurol. 214, 170–197. doi: 10.1002/cne.902140206

PubMed Abstract | CrossRef Full Text | Google Scholar

Moroso, A., Ruet, A., Lamargue-Hamel, D., Munsch, F., Deloire, M., Coupé, P., et al. (2017). Posterior lobules of the cerebellum and information processing speed at various stages of multiple sclerosis. J. Neurol. Neurosurg. Psychiatry 88, 146–151. doi: 10.1136/jnnp-2016-313867

PubMed Abstract | CrossRef Full Text | Google Scholar

Nordberg, A. (1993). Clinical studies in Alzheimer patients with positron emission tomography. Behav. Brain Res. 57, 215–224. doi: 10.1016/0166-4328(93)90138-g

PubMed Abstract | CrossRef Full Text | Google Scholar

Okonkwo, O. C., Crowe, M., Wadley, V. G., and Ball, K. (2008). Visual attention and self-regulation of driving among older adults. Int. Psychogeriatr. 20, 162–173. doi: 10.1017/S104161020700539X

PubMed Abstract | CrossRef Full Text | Google Scholar

Owen, A. M. (1997). Cognitive planning in humans: neuropsychological, neuroanatomical and neuropharmacological perspectives. Prog. Neurobiol. 53, 431–450. doi: 10.1016/s0301-0082(97)00042-7

PubMed Abstract | CrossRef Full Text | Google Scholar

Price, C. C., Cunningham, H., Coronado, N., Freedland, A., Cosentino, S., Penney, D. L., et al. (2011). Clock drawing in the Montreal Cognitive Assessment: recommendations for dementia assessment. Dement. Geriatr. Cogn. Disord. 31, 179–187. doi: 10.1159/000324639

PubMed Abstract | CrossRef Full Text | Google Scholar

Robbins, T. (2002). The 5-choice serial reaction time task: behavioural pharmacology and functional neurochemistry. Psychopharmacology 163, 362–380. doi: 10.1007/s00213-002-1154-7

PubMed Abstract | CrossRef Full Text | Google Scholar

Ruberg, M., Mayo, W., Brice, A., Duyckaerts, C., Hauw, J., Simon, H., et al. (1990). Choline acetyltransferase activity and (3H) vesamicol binding in the temporal cortex of patients with Alzheimer’s disease, Parkinson’s disease and rats with basal forebrain lesions. Neuroscience 35, 327–333. doi: 10.1016/0306-4522(90)90086-j

PubMed Abstract | CrossRef Full Text | Google Scholar

Rye, D., Wainer, B., Mesulam, M.-M., Mufson, E., and Saper, C. (1984). Cortical projections arising from the basal forebrain: a study of cholinergic and noncholinergic components employing combined retrograde tracing and immunohistochemical localization of choline acetyltransferase. Neuroscience 13, 627–643. doi: 10.1016/0306-4522(84)90083-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Sarter, M., and Paolone, G. (2011). Deficits in attentional control: cholinergic mechanisms and circuitry-based treatment approaches. Behav. Neurosci. 125:825. doi: 10.1037/a0026227

PubMed Abstract | CrossRef Full Text | Google Scholar

Schmahmann, J. D., and Sherman, J. C. (1998). The cerebellar cognitive affective syndrome. Brain 121, 561–579. doi: 10.1093/brain/121.4.561

PubMed Abstract | CrossRef Full Text | Google Scholar

Souillard-Mandar, W., Davis, R., Rudin, C., Au, R., Libon, D. J., Swenson, R., et al. (2016). Learning classification models of cognitive conditions from subtle behaviors in the digital clock drawing test. Mach. Learn. 102, 393–441. doi: 10.1007/s10994-015-5529-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Stern, Y. (2009). Cognitive reserve. Neuropsychologia 47, 2015–2028. doi: 10.1016/j.neuropsychologia.2009.03.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Tzourio-Mazoyer, N., Landeau, B., Papathanassiou, D., Crivello, F., Etard, O., Delcroix, N., et al. (2002). Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. Neuroimage 15, 273–289. doi: 10.1006/nimg.2001.0978

PubMed Abstract | CrossRef Full Text | Google Scholar

Valenstein, E., Watson, R. T., Van Den Abell, T., Carter, R., and Heilman, K. M. (1987). Response time in monkeys with unilateral neglect. Arch. Neurol. 44, 517–520. doi: 10.1001/archneur.1987.00520170045020

PubMed Abstract | CrossRef Full Text | Google Scholar

Vogels, O., Broere, C., Ter Laak, H., Ten Donkelaar, H., Nieuwenhuys, R., Schulte, B., et al. (1990). Cell loss and shrinkage in the nucleus basalis Meynert complex in Alzheimer’s disease. Neurobiol. Aging 11, 3–13. doi: 10.1016/0197-4580(90)90056-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Welsh, K. A., Breitner, J. C., and Magruder-Habib, K. M. (1993). Detection of dementia in the elderly using telephone screening of cognitive status. Neuropsychiatr. Neuropsychol. Behav. Neurol. 6, 103–110.

Google Scholar

White, M. G., and Mathur, B. N. (2018). Frontal cortical control of posterior sensory and association cortices through the claustrum. Brain Struct. Funct. 223, 2999–3006. doi: 10.1007/s00429-018-1661-x

PubMed Abstract | CrossRef Full Text | Google Scholar

Whitehouse, P. J., Price, D. L., Clark, A. W., Coyle, J. T., and Delong, M. R. (1981). Alzheimer disease: evidence for selective loss of cholinergic neurons in the nucleus basalis. Ann. Neurol. 10, 122–126. doi: 10.1002/ana.410100203

PubMed Abstract | CrossRef Full Text | Google Scholar

Whitfield-Gabrieli, S., and Nieto-Castanon, A. (2012). Conn: a functional connectivity toolbox for correlated and anticorrelated brain networks. Brain Connect. 2, 125–141. doi: 10.1089/brain.2012.0073

PubMed Abstract | CrossRef Full Text | Google Scholar

Worsley, K. J., Marrett, S., Neelin, P., Vandal, A. C., Friston, K. J., Evans, A. C., et al. (1996). A unified statistical approach for determining significant signals in images of cerebral activation. Hum. Brain Mapp. 4, 58–73. doi: 10.1002/(SICI)1097-0193(1996)4:1<58::AID-HBM4>3.0.CO;2-O

PubMed Abstract | CrossRef Full Text | Google Scholar

Yamamoto, S., Mogi, N., Umegaki, H., Suzuki, Y., Ando, F., Shimokata, H., et al. (2004). The clock drawing test as a valid screening method for mild cognitive impairment. Dement. Geriatr. Cogn. Disord. 18, 172–179. doi: 10.1159/000079198

PubMed Abstract | CrossRef Full Text | Google Scholar

Zaborszky, L., Hoemke, L., Mohlberg, H., Schleicher, A., Amunts, K., Zilles, K., et al. (2008). Stereotaxic probabilistic maps of the magnocellular cell groups in human basal forebrain. Neuroimage 42, 1127–1141. doi: 10.1016/j.neuroimage.2008.05.055

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhang, C., Zhou, P., and Yuan, T. (2016). The cholinergic system in the cerebellum: from structure to function. Rev. Neurosci. 27, 769–776. doi: 10.1515/revneuro-2016-0008

PubMed Abstract | CrossRef Full Text | Google Scholar