Neuropsychiatric Profile as a Predictor of Cognitive Decline in Mild Cognitive Impairment

Introduction: Mild cognitive impairment is often associated with affective and other neuropsychiatric symptoms (NPS). This co-occurrence might have a relevant impact on disease progression, from MCI to dementia.

Objective: The aim of this study was to explore the trajectories of cognitive decline in an MCI sample from a memory clinic, taking into consideration a perspective of isolated cognitive functions and based on NPS clusters, accounting for the different comorbid symptoms collected at their baseline visit.

Methods: A total of 2,137 MCI patients were monitored over a 2.4-year period. Four clusters of NPS (i.e., Irritability, Apathy, Anxiety/Depression and Asymptomatic) were used to run linear mixed models to explore the interaction of cluster with time on cognitive trajectories using a comprehensive neuropsychological battery (NBACE) administered at baseline and at the three subsequent follow-ups.

Results: A significant interaction between cluster and time in cognitive decline was found when verbal learning and cued-recall were explored (p = 0.002 for both memory functions). For verbal learning, the Irritability cluster had the largest effect size (0.69), whereas the Asymptomatic cluster showed the smallest effect size (0.22). For cued-recall, the Irritability cluster had the largest effect size among groups (0.64), and Anxiety/Depression had the smallest effect size (0.21).

Conclusions: In MCI patients, the Irritability and Apathy NPS clusters shared similar patterns of worsening in memory functioning, which could point to these NPS as risk factors of a faster cognitive decline, acting as early prognostic markers and helping in the diagnostic process.

Introduction

Biological changes bonded to impairment of cognitive functions are shown as humans age (Glisky, 2007). In the elderly, some cognitive skills such as attention, memory, executive functions or processing speed suffer from subtle changes associated with the normal aging process (Park et al., 2002; Park and Reuter-Lorenz, 2009), whereas others suffer a greater cognitive decline beyond expected, but not all decrement in cognitive functioning in this population is a precursor of disease. Therefore it is important to distinguish between normal and pathological cognitive decline, mainly because it could affect the patient’s daily functioning (Ginsberg et al., 2019) worsening their quality of life. The accurate measurement of cognitive decline over time is of utmost importance as it could help in the diagnosis and posterior prognosis of different neurodegenerative diseases and other syndromes (Grober et al., 2008; Wise et al., 2019).

Cognitive impairment is often associated with affective symptoms, such as anxiety or depression (Geda et al., 2008; Hermida et al., 2012; Singh-Manoux et al., 2017), which have been widely reported in different populations. While more is known about the former in relation to cognitive decline (Gonzales et al., 2017), there is still not much agreement about other neuropsychiatric symptoms (NPS) that could interfere with or relate somehow to a worsening in neuropsychological measures over time in early stages of different diseases. Some studies demonstrate the co-existence of both factors, with NPS being the predecessors of cognitive decline, often for many years (Wise et al., 2019; Tsunoda et al., 2020). There is no consensus on the order of appearance of both neural insults; previously, it was thought that cognitive deficits were the main reason for medical consultation, while studies increasingly claimed that NPS were the precursors initially detected before any cognitive decline is shown (Mortby and Anstey, 2015; Ismail et al., 2018). In any event, it is important to delve into early cognitive decline and try to elucidate the factors favoring it. At this early stage, another important feature to keep in mind is that comorbid NPS are often found in the clinical practice, and this co-occurrence of NPS and cognitive decline might have a cumulative effect on disease progression (Geda et al., 2013). Many attempts have been made to identify specific profiles of NPS associated with Alzheimer’s disease (AD). Some studies have explored the existence of neuropsychiatric subsyndromes or the genetics of NPS that could be the basis of AD, but no clear conclusions have been raised so far (Canevelli et al., 2013; Huang et al., 2020). A high prevalence of NPS in AD has commonly been associated with a worsening in the patient’s functionality (Karttunen et al., 2011).

It is well known that NPS seem to play a critical role in early clinical stages of the dementia continuum (Karttunen et al., 2011; Burhanullah et al., 2020), such as in Mild Cognitive Impairment (MCI) (Lyketsos et al., 2002; Geda et al., 2008; Peters et al., 2012). In a search of profiles of clustered symptoms that could serve as markers of disease progression in early stages, NPS would act as early clinical manifestations of an emergent process of neurodegeneration (Gallagher et al., 2017). In particular, affective NPS (depression, apathy, anxiety and irritability) were associated with a more rapid progression to AD in older adults with MCI (Jang et al., 2020), and those have also even shown synergic effects with the APOE ε4-allele (Valero et al., 2020). Recently, some attempts have been made to investigate grouped NPS as possible predictors of cognitive decline along the progression of MCI toward dementia (Palmer et al., 2007; Edwards et al., 2009). In a recent 2-year prospective study, and according to the three classes found in terms of NPS trajectories (stable, improved and worsened) in MCI patients, it was found that the NPS worsened class suffered the greatest cognitive and functional decline, as well as the highest conversion rate in comparison with the stable class and the improved class (David et al., 2016). Other clinical studies exploring associations of NPS by using factor analysis in MCI and mild AD dementia were focused on conversion to dementia and/or its relation to the severity of cognitive decline, but not specific cognitive domains (Siafarikas et al., 2018; Liew, 2019). There are two studies in the same line exploring NPS clusters and conversion to dementia in cognitively healthy volunteers (Leoutsakos et al., 2015; Forrester et al., 2016). However, there is still no consensus in the findings, probably due to dissimilarities in the design and methodology of these studies (different diagnostic criteria, sample selection or neuropsychological assessment applied) (Ma, 2020). Likewise, there is a conceptual void when exploring the most common NPS in patients with MCI and their implications in cognitive decline in a long-term follow-up to analyze patients’ progression in specific domains. Only a few studies in neurological patients, such as those with Parkinson’s and Huntington’s diseases (Pirogovsky-Turk et al., 2017), are going in this direction of assessing NPS in the MCI population and their implications for cognitive decline (Weintraub et al., 2015; Donaghy et al., 2018).

Therefore, this longitudinal study aims at investigating the existence of different trajectories of specific cognitive-domain decline over time in an MCI sample from a memory clinic, considering baseline NPS clustering.

Materials and Methods

Participants

The study was conducted at the Memory Clinic of ACE Alzheimer Center Barcelona (Spain), a private non-profit institution focused on the diagnosis, care and research of cognitive disorders and providing services to the Catalan Public Health Service (Xarxa Hospitalària d’Utilització Pública, XHUP) (Boada et al., 2014).

A total of 2,137 patients diagnosed with MCI were selected from a pool of patients evaluated at the Memory Clinic, see Roberto et al. (2021) for more information; MCI subtypes diagnoses were based on modified Petersen’s criteria and Lopez and colleagues’ classification, defined as amnestic (aMCI) or non-amnestic (naMCI), and possible or probable MCI due to AD, respectively (Petersen et al., 1999; Lyketsos et al., 2002; Petersen, 2004). All patients had to fulfill the following inclusion criteria: (i) more than 44 years old; (ii) a Mini-Mental State Examination (MMSE) total score of 24 or above; (iii) a Clinical Dementia Rating (CDR) score of 0.5; (iv) a Global Deterioration Scale (GDS) score of three or below; (v) at least six total years of formal education; (vi) absence of severe visual or auditory disturbances that could hinder the neuropsychological examination; (vii) presence of an informant or relative to complete the baseline administration of the NPI-Q; and, (viii) a baseline neuropsychological visit completed along with at least one follow-up. All clinical data were collected from January 2006 to June 2017. In all cases, the date of the MCI diagnosis was taken as the starting point or inclusion date for this study. Patients were followed up approximately annually with a clinical assessment that included a neurology and a neuropsychological visit.

Cognitive Measures

Cognitive data were collected at baseline and at every follow-up visit, using The Neuropsychological Battery of Fundació ACE (NBACE). The NBACE is a 50-min battery designed to assess cognitive domains especially affected in the elderly when dementia due to AD or other neurodegenerative processes is suspected (Alegret et al., 2012). The NBACE was proposed as a brief, easy-to-administer and goal-directed compilation of globally-used neuropsychological tests in our target population, provided that it is focused on verbal memory and learning, visual perception, and executive functions, which are affected early in the course of the disease. However, these are not the only explored domains. In our study we included tests sensitive to the following cognitive domains: attention, working memory, processing speed, executive functions, verbal memory, language, gnosis, visuospatial skills and praxis. Normative data and cut-off scores of the NBACE subtests for individuals more than 44 years old can be found elsewhere (Alegret et al., 2013).

Processing speed was measured with the Automatic Inhibition subtest of the Syndrome Kurtz Test (SKT; Erzigkeit, 1989), using execution time as the raw score. Attention and working memory scores were obtained by means of the digit span forward and backward subtests of the (WAIS–III; Wechsler, 1997a). Verbal learning and memory were measured through the word list learning test from the Wechsler Memory Scale–Third Edition. Verbal learning trials, long-term retention and cued-recall were used as raw scores; the interference list was not included in the battery (WMS–III; Wechsler, 1997b). Verbal learning was composed of the sum of raw scores obtained in the four trials of the learning phase (Σ1^st + 2^nd + 3^rd + 4^th); long-term retention was the total amount free recalled words; and cued-recall was the total number of words correctly recognized among the correct items and the same amount of “interference” items. Language was measured with the 15-item version of the Boston Naming Test (BNT; Kaplan et al., 1983). Gnosis, with a single score, was evaluated by means of the Poppelreuter test (Della Sala et al., 1995). Visuoconstructive praxis was evaluated with the abbreviated block design subtest of the Wechsler Adult Intelligence Scale–Third Edition WAIS–III (Wechsler, 1997a). Visuospatial skills were measured with Luria’s Clock Test (Golden, 1980), providing a single score. Finally, executive functioning was measured through different tests: the Automatic Inhibition of the SKT accuracy score of inhibition ability; phonetic, semantic and verb fluencies, obtaining three scores derived from the number of words recalled; and the abbreviated similarities subtest of the WAIS-III for abstract reasoning.

Neuropsychiatric Symptoms Measures

Neuropsychiatric symptoms were evaluated at the baseline clinical assessment using the Neuropsychiatric Inventory-Questionnaire (NPI-Q) (Boada et al., 2002). The NPI-Q is a simplified and widely-used scale that assesses 12 behavioral disturbances including delusions, hallucinations, agitation/aggression, depression/dysphoria, anxiety, elation/euphoria, apathy/indifference, disinhibition, irritability/lability, aberrant motor behavior, sleep and night-time behaviors, and appetite and eating disorders, in the dementia-related population. The NPI-Q was completed through information provided by a patient’s reliable informant (family member or caregiver). A change during the previous month in each one of the 12 behavioral domains was recorded as a dichotomized measure (present or absent). For more details on this measure and procedures see Roberto et al. (2021).

Analytical Approach

The present study is based on the results of a previous Latent Class Analysis (LCA) for clustering participants by means of a dichotomized NPI-Q measure (Roberto et al., 2021). Each participant was assigned to the best fitting cluster with the highest membership probability using baseline NPS. A 4-cluster was considered the optimal solution: Class 1 = Irritability; Class 2 = Apathy; Class 3 = Anxiety/Depression; Class 4 = Asymptomatic. Then, linear mixed-effects models (LMMs) were executed to explore cognitive decline for specific domains including NPS clusters and time of assessment (four time points, baseline, 1−, 2−, and 3-year follow-ups). Individual LMM models were calculated, one for each neuropsychological domain explored. Interaction of NPS cluster by time of assessment was considered the main effect of interest in the model, also incorporating the corresponding cluster and time main effects. For these two factors, fix and also as a random effect were analyzed, considering that assessment time points could vary among participants, and they also had different conditional probabilities of cluster belonging (see Roberto et al., 2021), i.e., individual differences had to be modeled. Mean differences (SD) accounting for time between baseline and every follow-up (times of assessments) were as it follows: from baseline to the first follow up were 11.26 months (6.22); from baseline to the second follow-up were 22 months (7.48); and from baseline to the third follow-up were 32.21 months (9.01). Both random intercept and slopes were included in the analyses. Asymptomatic class was considered the reference category. As a controlling factors, age, MMSE, educational level, sex, conversion to dementia (yes/no), MCI type (amnestic/non-amnestic), and MCI profile (possible/probable) were also included in the models and were considered in the models as fixed factors. Only when a significant interaction (cluster × time) was obtained in a specific cognitive domain, simple effects were calculated contrasting differences among clusters across the time points. Syntax of LMM is provided in Supplementary Material.

Results

According to previously published findings from our group (Roberto et al., 2021), the whole sample of 2,137 MCI patients was divided into four NPS clusters. Class 1-Irritability included 134 patients (6.3%) with high probability of irritability (0.93), together with lower probability of anxiety (0.64) and apathy (0.63). Class 2-Apathy comprised 272 patients (12.7%) and it was strongly represented by this symptom (1). Class 3-Anxiety/Depression included 1,056 patients (49.4%) who showed a high probability of depression (0.95), anxiety (0.93) and, by far, apathy (0.61). Class 4-Asymptomatic included 675 patients (31.6%) with low probabilities (<0.3) in all NPS.

Demographic and clinical characteristics of the sample stratified by NPS cluster are shown in Table 1. There were significant differences in age, gender, educational level, MMSE total score, MCI type, MCI profile, and conversion rates to dementia among the four NPS clusters, thus those were included in the models and were considered as controlling factors. Differences among groups in age distribution showed that Apathy patients were the oldest (mean age 76.2). In relation to gender, women were more prevalent in the Anxiety/Depression and Asymptomatic classes (65.2 and 60.9%, respectively). Educational level attained was higher in patients in the Apathy class (8.04 years of education). In relation to MMSE score obtained, the Anxiety/Depression class had the highest results (MMSE mean 27.02). Regarding the MCI type (amnestic vs. non-amnestic), our sample was quite balanced in general terms, having percentages in the four classes ranging from approximately 51 to 66%. According to the classification of possible or probable MCI profile, the Irritability and Apathy classes had a higher percentage of patients with a diagnosis of probable amnestic (47 and 48.2%, respectively). Finally, patients in the Irritability and Apathy classes showed a higher proportion of conversion to dementia.

TABLE 1

Table 1. Demographic characteristics and clinical variables of our final sample (n = 2137) stratified by neuropsychiatric symptoms cluster (NPS cluster).

Table 2 shows the LMM results for cognitive domains accounting for cluster, time, and cluster by time interaction. Only the memory domain showed a significant interaction in cluster by time, with verbal learning and cued-recall in particular being the only processes showing significant differences. Simple effects for these two cognitive functions comparing baseline with the third follow-up are displayed in Table 3, revealing significant results in all four clusters (p < 0.001). When differences between the final follow-up and the baseline scores were calculated in terms of verbal learning, a faster decline was shown in the Irritability class, with double the difference, versus a slower decline in the Asymptomatic class, with 3.66 and 1.34, respectively. With regard to effect size (Cohen’s D), the Irritability class had the highest score (0.69), whereas for Apathy and Anxiety/Depression classes the effect sizes were medium (0.49 and 0.31, respectively). The Asymptomatic class showed the smallest effect size (0.22). Similar results were obtained for cued-recall, but this time the Irritability and Apathy classes had quite similar differences between follow-up measures and the baseline (1.80 and 1.88, respectively), and the Anxiety/Depression class had the lowest score difference from the baseline. In terms of effect sizes, the Irritability class again showed the largest effect size (0.64), followed by the Apathy and Asymptomatic classes (0.46 and 0.30, respectively), with the Anxiety/Depression class having the smallest effect size (0.21). Cognitive decline for memory domains (learning and cued recall) were calculated for each cluster trajectory (see Figures 1, 2, respectively). Slopes of the trajectories were also presented for each cluster. Figures 1, 2 show speed of decline, with the Irritability class being the faster decliner (cognitive slope −0.98) and the Asymptomatic class the slower decliner (cognitive slope −0.43) in relation to verbal learning. For cued-recall, the Irritability and Apathy classes had the same cognitive slope value, and the Anxiety/Depression class showed a slower decline in this memory domain.

TABLE 2

Table 2. Linear mixed model results of cluster by time interaction and main effects in cognitive domains.

TABLE 3

Table 3. Simple effects and effect sizes of significant cognitive domains (i.e., verbal learning and cued-recall) between baseline (X₁) and third follow-up (X₄) measures.

FIGURE 1

Figure 1. Cognitive decline across clusters for verbal learning. Measures for each group were obtained using LMM means by calculating differences between baseline and follow-ups for each time point. Slopes for each cluster were calculated using the 2-known points approach. Negative values correspond to decrements: the larger the absolute values the steeper the line. Numbers at the end of the lines indicate the cognitive slopes for each class.

FIGURE 2

Figure 2. Cognitive decline across clusters for cued-recall. Measures for each group were obtained using LMM means by calculating differences between baseline and follow-ups for each time point. Slopes for each cluster were calculated using the 2-known points approach. Negative values correspond to decrements: the larger the absolute values the steeper the line. Numbers at the end of the lines indicate the cognitive slopes for each class.

Discussion

The findings in our study revealed different trajectories of cognitive decline in memory domains depending on NPS clusters (Irritability, Apathy, Anxiety/Depression, and Asymptomatic) in patients with MCI. In our sample of 2,137 MCI patients, the Irritability and Apathy NPS classes shared a similar pattern of faster cognitive decline in two memory domains (verbal learning and cued-recall), compared to the Anxiety/Depression and Asymptomatic classes, which showed a slower cognitive worsening over the stipulated follow-up period. Even though Irritability was the least prevalent neuropsychiatric condition in this sample, it proved to be the NPS class with the worst and fastest cognitive decline. Therefore, the present findings suggest that although Irritability and Apathy are less frequent NPS in MCI, these symptoms should be taken into account to improve the quality and usefulness in diagnostic and prognostic evaluation of cognitive worsening in MCI patients, especially those with an amnestic profile.

Although irritability is included among the so-called affective NPS, there is no substantial literature reporting consistent results on how individuals evolve in terms of cognitive decline, neither in healthy controls (Lobo et al., 2008; Leoutsakos et al., 2015), nor in patients with MCI (Forrester et al., 2016), and/or dementia (Moran et al., 2004). Indeed, most of the studies in MCI have focused on other affective NPS such as anxiety, apathy and depression (Penna, 2013). However, some authors have postulated that irritability could be among the affective symptoms that foretell a faster decline in conversion to dementia (Ismail et al., 2017; Jang et al., 2020), but to date none have provided data on that. Therefore, the present results partially support this hypothesis, adding some novelty about which cognitive domains could be more affected, always taking into account that the resulting clusters are mainly constellations of NPS, in which one symptom is the most manifest. For instance, Irritability cluster embraced irritable symptoms (0.93), but also anxiety and apathy to a lesser extent (both 0.63). Therefore, it is possible that the differential cognitive decline observed in individuals belonging to the Irritability cluster may be somehow influenced by anxious and apathic symptoms. In any case, with the present results, this is just speculative, but future studies should explore the mechanistic process underneath the effect of NPS on cognitive decline. It can be hypothesized that the presence of irritability may confer extra vulnerability to a faster conversion to dementia. It is worth mentioning that the large sample size of this study allowed the detection of an Irritability class, and it is probable that previous studies failed to detect a consistent cluster comprising individuals with irritability due to the lower prevalence compared to other affective NPS.

In contrast, several studies have explored the relationship between apathy and cognitive decline. Some authors indicated an increased risk of progression from MCI to AD when apathy was presented in isolation (Vicini Chilovi et al., 2009; Richard et al., 2012), whereas others postulated the risk was even higher when combined with depressive symptoms (Ruthirakuhan et al., 2019). Strikingly, low isolated depressive symptoms were not associated with cognitive decline (Richard et al., 2012). Conversely, another recent study demonstrated that both apathy and anxiety were associated with cognitive decline when presented comorbidly (Johansson et al., 2020). Our results converged with these findings, as we observed a sharper cognitive decline suffered by patients in the Apathy class compared to Anxiety/Depression. Given that cognitive decline is one of the factors favoring conversion to dementia and it was adjusted in our analyses, the findings shed light on the NPS profiles that could entail an earlier risk of conversion, and thus act as isolated markers.

With regard to anxiety and depression, both are among the most prevalent affective NPS in MCI patients (Lyketsos et al., 2002; Zhang et al., 2012), but their influence on cognitive decline is still controversial (Chan et al., 2011). Those symptoms have mostly been considered to be precursors of dementia, whereas only a few studies considered anxiety and depression to be a mere reaction to cognitive losses perceived by the patient (Simard et al., 2009; Di Iulio et al., 2010), which could be a consequence of and not an early marker for conversion to dementia. The present findings provided evidence of no clear association between anxious and depressive symptoms and faster cognitive decline; in contrast to other studies, both symptoms did not yield a worst prognosis in our sample. Note that the cognitive trajectory of the Anxiety/Depression class was comparable to the Asymptomatic class in terms of showing no consistent cognitive decline for those two clusters, as reported by other researchers (Ismail et al., 2017; Martin and Velayudhan, 2020). These findings may suggest that although anxiety and depression are the most frequently detected and known affective NPS in patients with cognitive decline, clinicians should bear in mind other affective NPS beyond anxious and depressed manifestations that could be more relevant in the progression to dementia.

The classification of individuals by symptomatic classes rather than isolated symptoms seems to be more useful and informative as it better reflects day-to-day reality in a memory clinic. Among the different studies exploring MCI populations grouped according to comorbid NPS, significant differences exist in obtained cluster solutions, probably due to the methodological approaches used. For instance, some works used a volunteer sample (Leoutsakos et al., 2015; Forrester et al., 2016; Jang et al., 2020), whereas others used clinical samples (David et al., 2016; Siafarikas et al., 2018; Liew, 2019). The statistical approach and designs were also different among studies (i.e., LCA vs. factor analysis, techniques that group individuals vs. grouping characteristics, respectively; or cross-sectional vs. longitudinal), which could have undermined the importance of taking affective NPS into consideration in diagnostic and prognostic evaluations. One of the abovementioned publications assessed cognitive decline across latent classes (David et al., 2016). However, the authors did not include affective NPS per se, but rather their severity, and they only evaluated memory and executive function domains, apart from the MMSE, to obtain a global cognitive measure. Therefore, the present study represents a step forward as cognitive decline was explored in assessing different cognitive processes.

A relevant finding of the present study is therefore that not all cognitive domains were affected equally at this early stage, but instead they behaved as isolated processes that showed subtle differences in cognitive decline when NPS classes were taken into account. Likewise, the results also revealed that an accurate assessment of MCI patients, contrary to previous work in already diagnosed dementia patients (Escudero et al., 2019), should cover cognitive performance by domains as well as NPS as these can guide the prognosis of MCI, especially now, when diagnosis can be sought earlier than ever. Interestingly, different cognitive trajectories were observed according to early NPS instead of neurological symptoms, which could help clinicians be aware of a possible diagnosis of dementia or other neurodegenerative diseases (Geda et al., 2013; Dietlin et al., 2019; Wise et al., 2019), and consider what is necessary to slow down progression of the illness, where possible.

There are also limitations to be considered in the current study. First, there are baseline differences in demographic characteristics and clinical variables among NPS clusters which could undermine the findings, even though the analyses included these variables. Second, it is important to consider including medication records in future studies, as it could affect the evolution of an underlying neuropsychiatric condition. Also, a longer follow-up would be appropriate to determine how NPS and cognitive decline will interact in the long run, as well as to analyze the long-term stability of NPS classes. Also, the presence of early AD-related biomarkers would help achieve a more accurate etiological diagnosis, and to benchmark NPS observations. A third limitation is associated with the estimated variances of parameter estimates; these may have been biased because heteroscedasticity was not taken into account for the repeated measurements of individuals and consequently may have affected the precision of estimating the appropriate model. However, similar studies published so far have failed to account for heteroscedasticity and the findings are consistent. Finally, the last limitation is focused on sex perspective. In the present study it was necessary to explore over 2,100 MCI participants to generate a consistent group of 134 irritable participants. In fact, this is one of the most relevant findings of our previous study (Roberto et al., 2021). Unfortunately, in this cluster, only 39 were women. The idea to analyze data considering sex as a main factor (in interaction with cluster and time) is, of course, clinically interesting, but the distribution of this factor in the different clusters, and especially in the irritable group, prevented us to consider sex factor as a key effect. More longitudinal studies are necessary, with larger samples, to obtain consistent empirical groups of patients where cognitive trajectories in men and women could be estimated with precision.

Conclusion

The approach of this study explores specific cognitive decline trajectories based on affective NPS clusters in MCI patients from a memory clinic, adding some novelty with respect to previous works. Specifically, and according to our results, Irritability and Apathy classes share a similar pattern of faster cognitive decline in two memory domains (verbal learning and cued-recall), compared to the Anxiety/Depression and Asymptomatic classes. The present findings emphasize the relevance of including an assessment of affective NPS when starting a diagnostic process provided that such symptoms—and in particular irritability and apathy—might act as aggravating factors. Our findings appear to open a new avenue to use NPS assessment as a clinical tool of great value when it comes to detecting in advance which patients could suffer from a marked worsening in cognition.

Data Availability Statement

The data that support this study may be provided upon reasonable request. Requests to access the datasets should be directed to SV, svalero@fundacioace.org.

Ethics Statement

The studies involving human participants were reviewed and approved by Hospital Clínic i Provincial of Barcelona, Spain. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

NR, MP, LT, MB, and SV conceived the idea, designed the study, and wrote the protocol and methodology. MM, MA, IH, AM, MR-R, CA, EE, SM-G, JT, LV, AE, GO, AP-C, ÁS, AO, IR, SM-G, and LM acquired all the data, performed technical procedures, and managed the data set. NR, MM, and AR managed previous literature searches. NR, MP, and SV contributed to the statistical analysis, interpretation of the results and writing of the first draft of the manuscript. EA-M helped in the statistical analysis and interpretation of the results. MM, MA, AR, LT, and MB revised the manuscript critically for intellectual content. All authors contributed to the writing of the final version and approved the manuscript.

Funding

MP receives funding from the Centro de Investigación Biomédica en Red de Salud Mental (CIBERSAM) and from the Generalitat de Catalunya through recognition of consolidate group or research (SGR17/001343). MM received research support from the Acción Estratégica en Salud, integrated in the Spanish National RCDCI Plan and financed by the Instituto de Salud Carlos III (ISCIII)—Subdirección General de Evaluación and the Fondo Europeo de Desarrollo Regional (FEDER—“Una manera de hacer Europa”) grant PII9/00335. AR has received support from CIBERNED [Instituto de Salud Carlos III (ISCIII)], the EU/EFPIA Innovative Medicines Initiative Joint Undertaking, ADAPTED Grant No. 115975, from the EXIT project, and EU Euronanomed3 Program JCT2017 Grant No. AC17/00100, from PREADAPT project. Joint Program for Neurodegenerative Diseases (JPND) Grant No. AC19/00097, and from grants PI13/02434, PI16/01861 BA19/00020, and PI19/01301, and Acción Estratégica en Salud, integrated in the Spanish National RCDCI Plan and financed by Instituto de Salud Carlos III (ISCIII)—Subdirección General de Evaluación and the Fondo Europeo de Desarrollo Regional (FEDER—“Una manera de Hacer Europa”), by Fundación Bancaria “La Caixa” and Grífols SA (GR@ACE project). MB has received grants from La Caixa, IMI, and ISCIII. MB has also received funding from Grifols, Biogen, Roche, Merck, Lilly, Nutricia, Cortexyme, Renew Research, Araclon. The funders are not involved in the study design, collection, analysis, interpretation of data, the writing of this article or the decision to submit it for publication.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

The authors would like to thank all participating patients and their caregivers for their essential contribution to this project.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnagi.2021.718949/full#supplementary-material

References

Alegret, M., Espinosa, A., Valero, S., Vinyes-Junqué, G., Ruiz, A., Hernández, I., et al. (2013). Cut-off scores of a brief neuropsychological battery (NBACE) for spanish individual adults older than 44 years old. PLoS One 8, 1–8. doi: 10.1371/journal.pone.0076436

PubMed Abstract | CrossRef Full Text | Google Scholar

Alegret, M., Espinosa, A., Vinyes-Junqué, G., Valero, S., Hernández, I., Tárraga, L., et al. (2012). Normative data of a brief neuropsychological battery for Spanish individuals older than 49. J. Clin. Exp. Neuropsychol. 34, 209–219. doi: 10.1080/13803395.2011.630652

PubMed Abstract | CrossRef Full Text | Google Scholar

Boada, M., Cejudo, J. C., Tàrraga, L., López, O. L., and Kaufer, D. (2002). Neuropsychiatric inventory questionnaire (NPI-Q): validación española de una forma abreviada del Neuropsychiatric Inventory (NPI). Neurología 17, 317–323.

Google Scholar

Boada, M., Tárraga, L., Hernández, I., Valero, S., Alegret, M., Ruiz, A., et al. (2014). Design of a comprehensive Alzheimer’s disease clinic and research center in Spain to meet critical patient and family needs. Alzheimers Dement. 10, 409–415. doi: 10.1016/j.jalz.2013.03.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Burhanullah, M. H., Tschanz, J. A. T., Peters, M. E., Leoutsakos, J. M., Matyi, J., Lyketsos, C. G., et al. (2020). Neuropsychiatric symptoms as risk factors for cognitive decline in clinically normal older adults: the cache county study. Am. J. Geriatr. Psychiatry 28, 64–71. doi: 10.1016/j.jagp.2019.03.023

PubMed Abstract | CrossRef Full Text | Google Scholar

Canevelli, M., Adali, N., Voisin, T., Soto, M. E., Bruno, G., Cesari, M., et al. (2013). Behavioral and psychological subsyndromes in Alzheimer’s disease using the neuropsychiatric inventory. Int. J. Geriatr. Psychiatry 28, 795–803. doi: 10.1002/gps.3904

PubMed Abstract | CrossRef Full Text | Google Scholar

Chan, W. C., Lam, L. C. W., Tam, C. W. C., Lui, V. W. C., Leung, G. T. Y., Lee, A. T. C., et al. (2011). Neuropsychiatric symptoms are associated with increased risks of progression to dementia: a 2-year prospective study of 321 Chinese older persons with mild cognitive impairment. Age Ageing 40, 30–35. doi: 10.1093/ageing/afq151

PubMed Abstract | CrossRef Full Text | Google Scholar

David, N. D., Feng Lin, R. N., and Porsteinsson, A. P. (2016). Trajectories of neuropsychiatric symptoms and cognitive decline in mild cognitive impairment Nicholas. Am. J. Geriatr. Psychiatry 24, 70–80. doi: 10.1016/j.jagp.2015.06.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Della Sala, S., Laiacona, M., Trivelli, C., and Spinnler, H. (1995). Poppelreuter-Ghent overlapping figures test. Its sensitivity to age, and its clinical use. Arch. Clin. Neuropsychol. 10, 511–534.

Google Scholar

Di Iulio, F., Palmer, K., Blundo, C., Casini, A. R., Gianni, W., Caltagirone, C., et al. (2010). Occurrence of neuropsychiatric symptoms and psychiatric disorders in mild Alzheimer’s disease and mild cognitive impairment subtypes. Int. Psychogeriatrics 22, 629–640. doi: 10.1017/S1041610210000281

PubMed Abstract | CrossRef Full Text | Google Scholar

Dietlin, S., Soto, M., Kiyasova, V., Pueyo, M., De Mauleon, A., Delrieu, J., et al. (2019). Neuropsychiatric symptoms and risk of progression to Alzheimer’s disease among mild cognitive impairment subjects. J. Alzheimers Dis. 70, 25–34. doi: 10.3233/JAD-190025

PubMed Abstract | CrossRef Full Text | Google Scholar

Donaghy, P. C., Taylor, J. P., O’Brien, J. T., Barnett, N., Olsen, K., Colloby, S. J., et al. (2018). Neuropsychiatric symptoms and cognitive profile in mild cognitive impairment with lewy bodies. Psychol. Med. 48, 2384–2390. doi: 10.1017/S0033291717003956

PubMed Abstract | CrossRef Full Text | Google Scholar

Edwards, E. R., Spira, A. P., Barnes, D. E., and Yaffe, K. (2009). Neuropsychiatric symptoms in mild cognitive impairment: differences by subtype and progression to dementia. Int. J. Geriatr. Psychiatry 24, 716–722. doi: 10.1002/gps.2187

PubMed Abstract | CrossRef Full Text | Google Scholar

Erzigkeit, H. (1989). “The SKT: A short cognitive performance test as an instrument for the assessment of clinical efficacy of cognitive enhancers,” in Diagnosis And Treatment Of Senile Dementia, eds M. Berenger and B. Reisberg (Berlin: Springer Verlag), 164–174.

Google Scholar

Escudero, J. M. S., Beltrán, J., Palacios, Á, Chimbí, C. M., Matallana, D., Reyes, P., et al. (2019). Neuropsychiatric symptoms as predictors of clinical course in neurodegeneration. A longitudinal study. Front. Aging Neurosci. 11:176. doi: 10.3389/fnagi.2019.00176

PubMed Abstract | CrossRef Full Text | Google Scholar

Forrester, S. N., Gallo, J. J., Smith, G. S., and Leoutsakos, J.-M. (2016). Patterns of neuropsychiatric symptoms in MCI and risk of dementia. Am. J. Geriatr. Psychiatry 24, 87–92. doi: 10.1016/j.coviro.2015.09.001.Human

CrossRef Full Text | Google Scholar

Gallagher, D., Fischer, C. E., and Iaboni, A. (2017). Neuropsychiatric symptoms in mild cognitive impairment: an update on prevalence, mechanisms, and clinical significance. Can. J. Psychiatry 62, 161–169. doi: 10.1177/0706743716648296

PubMed Abstract | CrossRef Full Text | Google Scholar

Geda, Y. E., Roberts, R. O., Knopman, D. S., Petersen, R. C., Christianson, T. J. H., Pankratz, V. S., et al. (2008). Prevalence of neuropsychiatric symptoms in mild cognitive impairment and normal cognitive aging: population-based study. Arch. Gen. Psychiatry 65, 1193–1198. doi: 10.1001/archpsyc.65.10.1193

PubMed Abstract | CrossRef Full Text | Google Scholar

Geda, Y. E., Schneider, L. S., Gitlin, L. N., Miller, D. S., Smith, G. S., Bell, J., et al. (2013). Neuropsychiatric symptoms in Alzheimer’s disease: past progress and anticipation of the future. Alzheimers Dement. 9, 602–608. doi: 10.1016/j.jalz.2012.12.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Ginsberg, T. B., Powell, L., Emrani, S., Wasserman, V., Higgins, S., Chopra, A., et al. (2019). Instrumental activities of daily living, neuropsychiatric symptoms, and neuropsychological impairment in mild cognitive impairment. J. Am. Osteopath. Assoc. 119, 96–101. doi: 10.7556/jaoa.2019.015

PubMed Abstract | CrossRef Full Text | Google Scholar

Glisky, E. (2007). “Changes in cognitive function in human aging,” in Brain Aging, ed. D. R. Riddle (Boca Raton, FL: CRC Press/Taylor & Francis). doi: 10.1201/9781420005523.sec1

CrossRef Full Text | Google Scholar

Golden, C. J. (1980). In reply to adams “in search of luria’s battery: a false start”. J. Consult. Clin. Psychol. 48, 517–521.

Google Scholar

Gonzales, M. M., Insel, P. S., Nelson, C., Tosun, D., Mattsson, N., Mueller, S. G., et al. (2017). Cortical atrophy is associated with accelerated cognitive decline in mild cognitive impairment with subsyndromal depression. Am. J. Geriatr. Psychiatry 25, 980–991. doi: 10.1016/j.jagp.2017.04.011

PubMed Abstract | CrossRef Full Text | Google Scholar

Grober, E., Hall, C. B., Lipton, R. B., Zonderman, A. B., Resnick, S. M., and Kawas, C. (2008). Memory impairment, executive dysfunction, and intellectual decline in preclinical Alzheimer’s disease. J. Int. Neuropsychol. Soc. 14, 266–278.

Google Scholar

Hermida, A. P., McDonald, W. M., Steenland, K., and Levey, A. (2012). The association between late-life depression, mild cognitive impairment and dementia: is inflammation the missing link? Expert Rev. Neurother. 12, 1339–1350. doi: 10.1586/ern.12.127

PubMed Abstract | CrossRef Full Text | Google Scholar

Huang, M. F., Lee, W. J., Yeh, Y. C., Liao, Y. C., Wang, S. J., Yang, Y. H., et al. (2020). Genetics of neuropsychiatric symptoms in patients with Alzheimer’s disease: a 1-year follow-up study. Psychiatry Clin. Neurosci. 74, 645–651. doi: 10.1111/pcn.13150

PubMed Abstract | CrossRef Full Text | Google Scholar

Ismail, Z., Agüera-Ortiz, L., Brodaty, H., Cieslak, A., Cummings, J., Fischer, C. E., et al. (2017). The mild behavioral impairment checklist (MBI-C): a rating scale for neuropsychiatric symptoms in pre-dementia populations. J. Alzheimers Dis. 56, 929–938. doi: 10.3233/JAD-160979

PubMed Abstract | CrossRef Full Text | Google Scholar

Ismail, Z., Gatchel, J., Bateman, D. R., Barcelos-Ferreira, R., Chantillon, M., Jaeger, J., et al. (2018). Affective and emotional dysregulation as pre-dementia risk markers: exploring the mild behavioral impairment symptoms of depression, anxiety, irritability, and euphoria. Int. Psychogeriatrics 30, 185–196. doi: 10.1017/S1041610217001880

PubMed Abstract | CrossRef Full Text | Google Scholar

Jang, J. Y., Ho, J. K., Blanken, A. E., Dutt, S., and Nation, D. A. (2020). Affective neuropsychiatric symptoms as early signs of dementia risk in older adults. J. Alzheimers Dis. 77, 1195–1207. doi: 10.3233/jad-200190

PubMed Abstract | CrossRef Full Text | Google Scholar

Johansson, M., Stomrud, E., Lindberg, O., Westman, E., Johansson, P. M., van Westen, D., et al. (2020). Apathy and anxiety are early markers of Alzheimer’s disease. Neurobiol. Aging 85, 74–82. doi: 10.1016/j.neurobiolaging.2019.10.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Kaplan, E. F., Goodglass, H., and Weintraub, S. (1983). The Boston Naming Test. Philadelphia, PA: Lea & Febiger.

Google Scholar

Karttunen, K., Karppi, P., Hiltunen, A., Vanhanen, M., Välimäki, T., Martikainen, J., et al. (2011). Neuropsychiatric symptoms and quality of life in patients with very mild and mild Alzheimer’s disease. Int. J. Geriatr. Psychiatry 26, 473–482. doi: 10.1002/gps.2550

PubMed Abstract | CrossRef Full Text | Google Scholar

Leoutsakos, J. M. S., Forrester, S. N., Lyketsos, C. G., and Smith, G. S. (2015). Latent classes of neuropsychiatric symptoms in NACC controls and conversion to mild cognitive impairment or dementia. Handb. Depress. Alzheimers Dis. 48, 81–92. doi: 10.3233/978-1-61499-542-5-81

CrossRef Full Text | Google Scholar

Liew, T. M. (2019). Symptom clusters of neuropsychiatric symptoms in mild cognitive impairment and their comparative risks of dementia: a cohort study of 8530 older persons. J. Am. Med. Dir. Assoc. 20, 1054–1054.e9. doi: 10.1016/j.jamda.2019.02.012

PubMed Abstract | CrossRef Full Text | Google Scholar

Lobo, A., López-Antón, R., De-La-Cámara, C., Quintanilla, M. Á, Campayo, A., and Saz, P. (2008). Non-cognitive psychopathological symptoms associated with incident mild cognitive impairment and dementia, Alzheimer’s type. Neurotox. Res. 14, 263–272. doi: 10.1007/BF03033815

PubMed Abstract | CrossRef Full Text | Google Scholar

Lyketsos, G. C., López, O., Jones, B., Fitzpatrick, A. L., Breitner, J., and Dekosky, S. (2002). Prevalence of neuropsychiatric symptoms in dementia and mci: results from the cardiovascular health study. JAMA 288, 1475–1483. doi: 10.1001/jama.288.12.1475

PubMed Abstract | CrossRef Full Text | Google Scholar

Ma, L. (2020). Depression, anxiety, and apathy in mild cognitive impairment: current perspectives. Front. Aging Neurosci. 12:9. doi: 10.3389/fnagi.2020.00009

PubMed Abstract | CrossRef Full Text | Google Scholar

Martin, E., and Velayudhan, L. (2020). Neuropsychiatric symptoms in mild cognitive impairment: a literature review. Dement. Geriatr. Cogn. Disord. 49, 146–155. doi: 10.1159/000507078

PubMed Abstract | CrossRef Full Text | Google Scholar

Moran, M., Walsh, C., Lynch, A., Coen, R. F., Coakley, D., and Lawlor, B. A. (2004). Syndromes of behavioural and psychological symptoms in mild Alzheimer’s disease. Int. J. Geriatr. Psychiatry 19, 359–364. doi: 10.1002/gps.1091

PubMed Abstract | CrossRef Full Text | Google Scholar

Mortby, M. E., and Anstey, K. J. (2015). “Mental health and aging,” in Encyclopedia of Geropsychology ed. A. N. Pachana Singapore: Springer Singapore. doi: 10.1007/978-981-287-080-3_1-1

CrossRef Full Text | Google Scholar

Palmer, K., Berger, A. K., Monastero, R., Winblad, B., Bäckman, L., and Fratiglioni, L. (2007). Predictors of progression from mild cognitive impairment to Alzheimer disease. Neurology 68, 1596–1602. doi: 10.1212/01.wnl.0000260968.92345.3f

PubMed Abstract | CrossRef Full Text | Google Scholar

Park, D. C., and Reuter-Lorenz, P. (2009). The adaptive brain: aging and neurocognitive scaffolding. Annu. Rev. Psychol. 60, 173–196. doi: 10.1146/annurev.psych.59.103006.093656

PubMed Abstract | CrossRef Full Text | Google Scholar

Park, D. C., Lautenschlager, G., Hedden, T., Davidson, N. S., Smith, A. D., and Smith, P. K. (2002). Models of visuospatial and verbal memory across the adult life span. Psychol. Aging 17, 299–320. doi: 10.1037/0882-7974.17.2.299

CrossRef Full Text | Google Scholar

Penna, S. (2013). Cognitive and emotional dysfunction in mild cognitive impairment. Clin. Geriatr. Med. 29, 773–789. doi: 10.1016/j.cger.2013.07.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Peters, M. E., Rosenberg, P. B., Steinberg, M., Tschanz, J. T., Norton, M. C., Welsh-Bohmer, K. A., et al. (2012). Prevalence of neuropsychiatric symptoms in CIND and its subtypes: the cache county study. Am. J. Geriatr. Psychiatry 20, 416–424. doi: 10.1097/JGP.0b013e318211057d

PubMed Abstract | CrossRef Full Text | Google Scholar

Petersen, R. C. (2004). Mild cognitive impairment as a diagnostic entity. J. Intern. Med 256, 183–194. doi: 10.1111/j.1365-2796.2004.01388.x/pdf

CrossRef Full Text | Google Scholar

Petersen, R. C., Smith, G. E., Waring, S. C., Ivnik, R. J., Tangalos, E. G., and Kokmen, E. (1999). Mild cognitive impairment: clinical characterization and outcome. Arch. Neurol. 56, 303–309.

Google Scholar

Pirogovsky-Turk, E., Moore, R. C., Filoteo, J. V., Litvan, I., Song, D. D., Lessig, S. L., et al. (2017). Neuropsychiatric predictors of cognitive decline in parkinson disease: a longitudinal study. Am. J. Geriatr. Psychiatry 25, 279–289. doi: 10.1016/j.jagp.2016.10.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Richard, E., Schmand, B., Eikelenboom, P., Yang, S. C., Ligthart, S. A., Moll Van Charante, E. P., et al. (2012). Symptoms of apathy are associated with progression from mild cognitive impairment to Alzheimer’s disease in non-depressed subjects for the Alzheimer’s disease neuroimaging initiative. Dement. Geriatr. Cogn. Disord. 33, 204–209. doi: 10.1159/000338239

PubMed Abstract | CrossRef Full Text | Google Scholar

Roberto, N., Portella, M. J., Marquié, M., Alegret, M., Hernández, I., Mauleón, A., et al. (2021). Neuropsychiatric profiles and conversion to dementia in mild cognitive impairment, a latent class analysis. Sci. Rep. 11, 1–9. doi: 10.1038/s41598-021-83126-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Ruthirakuhan, M., Herrmann, N., Vieira, D., Gallagher, D., and Lanctôt, K. L. (2019). The roles of apathy and depression in predicting Alzheimer disease: a longitudinal analysis in older adults with mild cognitive impairment. Am. J. Geriatr. Psychiatry 27, 873–882. doi: 10.1016/j.jagp.2019.02.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Siafarikas, N., Selbaek, G., Fladby, T., Šaltyte Benth, J., Auning, E., and Aarsland, D. (2018). Frequency and subgroups of neuropsychiatric symptoms in mild cognitive impairment and different stages of dementia in Alzheimer’s disease. Int. Psychogeriatrics 30, 103–113. doi: 10.1017/S1041610217001879

PubMed Abstract | CrossRef Full Text | Google Scholar

Simard, M., Hudon, C., and van Reekum, R. (2009). Psychological distress and risk for dementia. Curr. Psychiatry Rep. 11, 41–47.

Google Scholar

Singh-Manoux, A., Dugravot, A., Fournier, A., Abell, J., Ebmeier, K., Kivimäki, M., et al. (2017). Trajectories of depressive symptoms before diagnosis of dementia: a 28-year follow-up study. JAMA Psychiatry 74, 712–718. doi: 10.1001/jamapsychiatry.2017.0660

PubMed Abstract | CrossRef Full Text | Google Scholar

Tsunoda, K., Yamashita, T., Osakada, Y., Sasaki, R., Tadokoro, K., Matsumoto, N., et al. (2020). Early emergence of neuropsychiatric symptoms in cognitively normal subjects and mild cognitive impairment. J. Alzheimers Dis. 73, 209–215. doi: 10.3233/JAD-190669

PubMed Abstract | CrossRef Full Text | Google Scholar

Valero, S., Marquié, M., De Rojas, I., Espinosa, A., Moreno-Grau, S., Orellana, A., et al. (2020). Interaction of neuropsychiatric symptoms with APOE ε4 and conversion to dementia in MCI patients in a memory clinic. Sci. Rep. 10, 1–10. doi: 10.1038/s41598-020-77023-z

PubMed Abstract | CrossRef Full Text | Google Scholar

Vicini Chilovi, B., Conti, M., Zanetti, M., Mazzù, I., Rozzini, L., and Padovani, A. (2009). Differential impact of apathy and depression in the development of dementia in mild cognitive impairment patients. Dement. Geriatr. Cogn. Disord. 27, 390–398. doi: 10.1159/000210045

PubMed Abstract | CrossRef Full Text | Google Scholar

Wechsler, D. (1997a). WAIS–III. Wechsler Adult Intelligence Scale–Third Edition. Technical Manual. San Antonio, TX: The Psychological Corporation.

Google Scholar

Wechsler, D. (1997b). WMS–III. Wechsler Memory Scale–Third Edition. Administration and Scoring Manual. San Antonio, TX: The Psychological Corporation.

Google Scholar

Weintraub, D., Simuni, T., Caspell-Garcia, C., Coffey, C., Lasch, S., Siderowf, A., et al. (2015). Cognitive performance and neuropsychiatric symptoms in early, untreated Parkinson’s disease. Physiol. Behav. 30, 919–927. doi: 10.1002/mds.26170

PubMed Abstract | CrossRef Full Text | Google Scholar

Wise, E. A., Rosenberg, P. B., Lyketsos, C. G., and Leoutsakos, J. M. (2019). Time course of neuropsychiatric symptoms and cognitive diagnosis in national Alzheimer’s coordinating centers volunteers. Alzheimers Dement. Diagnosis Assess. Dis. Monit. 11, 333–339. doi: 10.1016/j.dadm.2019.02.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhang, M., Wang, H., Li, T., and Yu, X. (2012). Prevalence of neuropsychiatric symptoms across the declining memory continuum: an observational study in a memory clinic setting. Dement. Geriatr. Cogn. Dis. Extra 2, 200–208. doi: 10.1159/000338410

PubMed Abstract | CrossRef Full Text | Google Scholar