Predicting AD conversion with a computerized diagnostic tool
-
1
University of Eastern Finland, Finland
-
2
VTT Technical Research Centre of Finland, Finland
-
3
Copenhagen University Hospital, Denmark
Purpose
To compare the accuracies of predicting AD conversion by using a decision support system (PredictAD tool 1) and current research criteria of prodromal AD as identified by combinations of episodic memory impairment of hippocampal type and visual assessment of medial temporal lobe atrophy (MTA) on MRI and CSF biomarkers.
Subjects and Methods
A total of 391 MCI cases were selected from the ADNI cohort (http://adni.loni.ucla.edu/). The demographics of the cases are summarized in Table 1. The definition of MCI is as follow: 1) subjects had Mini-Mental State Examination (MMSE) score between 24 and 30, 2) the memory complaint, 3) objective memory loss measured by education adjusted scores on Wechsler Memory Scale-Revised (WMS-R) Logical Memory II, 4) Clinical Dementia Rating (CDR) of 0.5, 5) the absence of significant levels of impairment in other cognitive domains, essentially preserved activities of daily living, and 6) the absence of dementia. All the cases had baseline ADNI cognitive testing results, including MMSE, Alzheimer's Disease Assessment Scale-Cognitive subscale (ADAS-Cog), and several other common neuropsychological tests (http://adni.loni.ucla.edu/).
Predicting AD conversion with current prodromal AD guidelines
The prediction of AD conversion was conducted with the combinations of clinical diagnosis of hippocampal pattern of memory loss [1] and biomarkers [2,3]. The episodic memory loss of the hippocampal type, which is characterized by a free recall deficit on testing not normalized with cueing [1], was defined as present when the scores of delayed recall and delayed recognition of Auditory Verbal Learning Test (RAVLT) [4] were lower than 1 standard deviation of the corresponding mean values in healthy aged people, i.e. RAVLT delayed recall < 3 and RAVLT delayed recognition < 10 [5]. The Scheltens Scale was used to categorize the visual medial temporal lobe atrophy (MTA) on MRI, The scale rates atrophy on a 5-point scale (0 = absent, 1 = minimal, 2 = mild, 3 = moderate and 4 = severe) [2]. A single experienced neuroradiologist (YL) evaluated MTA in all of the cases. Scheltens score ≥3 was considered as having significant MTA. CSF levels of Tau > 93 pg/ml, and Amyloid beta 1-42 (Aβ1-42) < 192pg/ml were considered as positive CSF markers [3]. The likelihood of AD conversion was defined as follows [1]:
• High likelihood: all clinical core criteria (RAVLT tests), Scheltens scale and CSF markers were positive,
• Intermediate likelihood: clinical core criteria was positive, one of MRI and CSF markers was positive, but the other one was lacking, i.e., not available,
• Uninformative likelihood: clinical core criteria was positive, and one of MRI and CSF markers was positive, but the other one was negative.
• Low likelihood: all clinical core criteria, Scheltens scale, and CSF markers were negative.
Predict conversion to AD with PredictAD tool
The PredictAD tool [6] was used by one clinician who was blinded to the outcome during the evaluation. The PredictAD tool provided the rater with the available patient information at baseline, including demographics, apolipoprotein E (APOE) genotype, MMSE, ADAS-Cog, neuropsychological battery, MRI features automatically derived with FreeSurfer software package, and CSF laboratory analysis results. In addition, several features automatically derived from original MRI images using manifold learning [7], tensor-based morphometry [8], and hippocampus volume segmentation [9], developed in the PredictAD project (www.predictad.eu), were included. When determining with the assistance of PredictAD tool whether a subject had prodromal AD, the clinician based his opinion on presence of abnormal performances in the delayed recall and delayed recognition of Auditory RAVLT, the other neuropsychological tests were used as supportive evidences to determine the confidence of the clinical diagnosis. Given the baseline data, the clinician was then asked to categorize, i.e. diagnose, each patient into one of six categories: 1) clear indication of Non-AD, 2) probable indication of Non-AD, 3) subtle indication of Non-AD, 4) subtle indication of early AD, 5) probable indication of early AD, and 6) clear indication of early AD. One must emphasize that the clinician was asked to predict the diagnostic outcomes (Non-AD and AD converter) at the end of ADNI study using exclusively baseline data. To compare the accuracy of classification between automatically computed PredictAD diagnosis and clinician’s diagnosis with assistance of PredictAD tool, Disease State Index (DSI) values, computed by the PredictAD tool, were categorized uniformly between 0 and 1 as follows: (1) Clear indication of Non-AD: DSI < 0.17 , (2) Probable indication of Non-AD: 0.17 ≤ DSI < 0.33, (3) Subtle indication of Non-AD: 0.33 ≤ DSI < 0.50, (4) Subtle indication of early AD: 0.50 ≤ DSI< 0.67, (5) Probable indication of early AD: 0.67 ≤ DSI < 0.83, and (6) Clear indication of early AD: ≥ 0.83. In the automatically computed PredictAD diagnosis, all the neuropsychological and genetic tests, MRI, and CSF data were used to calculate the DSI.
To test the reproducibility of the diagnosis by clinicians with the assistance of PredictAD tool, interobserver variability and intraobserver reproducibility were analyzed. To test the interobserver variability, two clinicians (Y.L. and M.M.) independently made diagnosis in 40 (10%) randomly selected cases. To test the intraobserver reproducibility, one clinician made diagnosis in the 40 cases with an interval of at least 6 months between the diagnosis sessions.
Results
A total 387 of 391 MCI cases had undergone MRI exams, 199 MCI cases had undergone CSF examination, and 195 MCI cases had both MRI and CSF exams. During the 3-year follow-up, 158 of 391 (40%) converted to AD, 15 of 391 (4%) returned to normal cognitive status, and 218 MCI cases (56%) remained stable.
Among the MCI cases who possessed a single positive marker (clinical core criteria or biomarker), those MCI cases who had increased Tau and decreased Aβ1-42 had the highest conversion rate (57%). The conversion rate for those MCI cases with Scheltens score≥3 was 55%. The MCI cases fulfilling the clinical core criteria for episodic memory loss evident both on free recall and recognition had the lowest conversion rate (53%).
As expected, the conversion rate was highest for those MCI subjects in high likelihood AD group (65%) and lowest for MCI subjects with low likelihood (7%). For the MCI cases with intermediate and uninformative likelihood of AD, the conversion rates were 57% and 64% respectively. Among the 20 baseline MCI cases estimated as high likelihood of AD, there were no significant differences in age, Scheltens score, concentrations of CSF Tau and Aβ1-42, AVLT scores, education years, gender, frequency of APOE e4 allele, or PredictAD DSI between converters (n=13) and non-converters (n=7) (p≥0.354).
Sensitivity, specificity, and accuracy using different criteria and PredictAD tool
The criteria of increased CSF Tau or decreased Aβ1-42 achieved the highest sensitivity (90%), but the lowest specificity (36%). The criteria that included episodic memory loss of the hippocampal type, Scheltens scale ≥3, increased CSF Tau, and decreases Aβ1-42 could correctly detect 111 of 115 non-AD converters, producing the highest specificity (98%), but the lowest sensitivity (6%).
The PredictAD tool produced the highest accuracy 72%, followed by the clinician’s diagnosis with the assistance of the PredictAD tool (71%). There was no significant difference in accuracy between the diagnosis by Predict tool alone and by the clinician (p=1.0). The accuracy of the diagnosis by PredictAD tool alone was significantly higher than if one used the criteria of the biomarkers alone or combinations of clinical diagnosis of hippocampal pattern of memory loss and biomarkers (p≤0.037).
When considering the six categories of diagnosis (from clear indication of early AD to clear indication of non-AD), the interobserver variability and intraobserver reproducibility showed moderate agreements (kappa=0.403, p<0.001; kappa=0.462, p<0.001, respectively). However, when we simplified the six categories of diagnosis into AD and non-AD groups, excellent agreements were achieved (kappa=0.800, p<0.001 for interobserver variability; kappa=0.850, p<0.001 for intraobserver reproducibility).
The PredictAD DSI achieved accuracy of 81% in detecting non-AD converters, and an accuracy of 63% in detecting AD converters. In the clinician’s diagnosis with the assistance of the PredictAD tool, the accuracies were 80% and 62% respectively. However, with the assistance of PredictAD tool, the clinician’s diagnosis of high confidence (clear non-AD, probable non-AD, probable AD, and clear AD) was dramatically improved compared to the PredictAD tool alone. The number of non-AD diagnoses made by the clinician with high confidence increased from 118 to 146 (from 30% to 37%), and the number of AD diagnosis with high confidence increased from 87 to 112 (from 22% to 29%). With help of the PredictAD tool, the clinician made diagnoses of clear non-AD or clear AD in 144 of 391 (37%) cases with overall accuracy of 84%.
The clear AD diagnoses (16 cases) in the PredictAD DSI index included 5 stable MCI cases. The Probable indication of AD (71 cases) in the PredictAD DSI index included 20 stable MCI individuals. Among this subgroup there were no significant differences in age, gender, presence of APOE 4, years of education, concentrations of CSF markers, Scheltens scores, MMSE, or RAVLT results between AD converters and those with stable MCI (p≥0.236).
Conclusion
With the assistance of the PredictAD tool, the clinician can predict AD conversion more accurately than than the current research criteria for prodromal AD.
Acknowledgements
Data used in the preparation of this article were obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database (www.loni.ucla.edu/ADNI). As such, the investigators within the ADNI contributed to the design and implementation of ADNI and/or provided data, but did not participate in analysis or writing of this report. A complete listing of ADNI investigators can be found at:
http://adni.loni.ucla.edu/wp-content/uploads/how_to_apply/ADNI_Acknowledgement_List.pdf.
References
1. Dubois B, Feldman HH, Jacova C, Cummings JL, Dekosky ST, et al. (2010) Revising the definition of alzheimer's disease: A new lexicon. Lancet Neurol 9: 1118-1127.
2. Scheltens P, Leys D, Barkhof F, Huglo D, Weinstein HC, et al. (1992) Atrophy of medial temporal lobes on MRI in "probable" alzheimer's disease and normal ageing: Diagnostic value and neuropsychological correlates. J Neurol Neurosurg Psychiatr 55: 967-972.
3. Shaw LM, Vanderstichele H, Knapik-Czajka M, Clark CM, Aisen PS, et al. (2009) Cerebrospinal fluid biomarker signature in alzheimer's disease neuroimaging initiative subjects. Ann Neurol 65: 403-413.
4. Rey A. (1964) L´examen clinique en psychologie. Paris: Presses Universitaires de France.
5. Ivnik RJ, Malec JF, Tangalos EG, Petersen RC, Kokmen E, et al. (1992) Mayo`s older americans normative studies: Updated AVLT norms for ages 56 to 97. The Clinical Neuropsychologist 6: 83-104.
6. Mattila J, Koikkalainen J, Virkki A, Simonsen A, van Gils M, et al. (2011) A disease state fingerprint for evaluation of alzheimer's disease. J Alzheimer's Dis 27: 163-176.
7. Wolz R, Aljabar P, Hajnal J, Rueckert D. (2010) Manifold learning for biomarker discovery in MR imaging. 6357: 116-123.
8. Koikkalainen J, Lotjonen J, Thurfjell L, Rueckert D, Waldemar G, et al. (2011) Multi-template tensor-based morphometry: Application to analysis of alzheimer's disease. Neuroimage 56: 1134-1144.
9. Lotjonen J, Wolz R, Koikkalainen J, Julkunen V, Thurfjell L, et al. (2011) Fast and robust extraction of hippocampus from MR images for diagnostics of alzheimer's disease. Neuroimage 56: 185-196.
Keywords:
AD,
MCI,
prodromal AD,
Diagnosis, Computer-Assisted,
MRI imaging,
Neuropsychological Tests,
CSF biomarkers,
APOE E4 allele
Conference:
Imaging the brain at different scales: How to integrate multi-scale structural information?, Antwerp, Belgium, 2 Sep - 6 Sep, 2013.
Presentation Type:
Poster presentation
Topic:
Poster session
Citation:
Liu
Y,
Mattila
J,
Paajanen
T,
Koikkalainen
J,
Van Gils
M,
Herukka
S,
Waldemar
G,
Lötjönen
J and
Soininen
H
(2013). Predicting AD conversion with a computerized diagnostic tool.
Front. Neuroinform.
Conference Abstract:
Imaging the brain at different scales: How to integrate multi-scale structural information?.
doi: 10.3389/conf.fninf.2013.10.00038
Copyright:
The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers.
They are made available through the Frontiers publishing platform as a service to conference organizers and presenters.
The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated.
Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed.
For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions.
Received:
23 Jul 2013;
Published Online:
31 Aug 2013.
*
Correspondence:
Dr. Yawu Liu, University of Eastern Finland, Kuopio, Finland, liuy88@hotmail.com
Prof. Hilkka Soininen, University of Eastern Finland, Kuopio, Finland, hilkka.soininen@kuh.fi