Assessment of the Concordance and Diagnostic Accuracy Between Elecsys and Lumipulse Fully Automated Platforms and Innotest

Manual ELISA assays are the most commonly used methods for quantification of biomarkers; however, they often show inter- and intra-laboratory variability that limits their wide use. Here, we compared the Innotest ELISA method with two fully automated platforms (Lumipulse and Elecsys) to determine whether these new methods can provide effective substitutes for ELISA assays. We included 149 patients with AD (n = 34), MCI (n = 94) and non-AD dementias (n = 21). Aβ42, T-tau, and P-tau were quantified using the ELISA method (Innotest, Fujirebio Europe), CLEIA method on a Lumipulse G600II (Fujirebio Diagnostics), and ECLIA method on a Cobas e 601 (Roche Diagnostics) instrument. We found a high correlation between the three methods, although there were systematic differences between biomarker values measured by each method. Both Lumipulse and Elecsys methods were highly concordant with clinical diagnoses, and the combination of Lumipulse Aβ42 and P-tau had the highest discriminating power (AUC 0.915, 95% CI 0.822–1.000). We also assessed the agreement of AT(N) classification for each method with AD diagnosis. Although differences were not significant, the use of Aβ42/Aβ40 ratio instead of Aβ42 alone in AT(N) classification enhanced the diagnostic accuracy (AUC 0.798, 95% CI 0.649–0.947 vs. AUC 0.778, 95% CI 0.617–0.939). We determined the cut-offs for the Lumipulse and Elecsys assays based on the Aβ42/Aβ40 ratio ± status as a marker of amyloid pathology, and these cut-offs were consistent with those recommended by manufacturers, which had been determined based on visual amyloid PET imaging or diagnostic accuracy. Finally, the biomarker ratios (P-tau/Aβ42 and T-tau/Aβ42) were more consistent with the Aβ42/Aβ40 ratio for both Lumipulse and Elecsys methods, and Elecsys P-tau/Aβ42 had the highest consistency with amyloid pathology (AUC 0.994, 95% CI 0.986–1.000 and OPA 96.4%) at the ≥0.024 cut-off. The Lumipulse and Elecsys cerebrospinal fluid (CSF) AD assays showed high analytical and clinical performances. As both automated platforms were standardized for reference samples, their use is recommended for the measurement of CSF AD biomarkers compared with unstandardized manual methods, such as Innotest ELISA, that have demonstrated a high inter and intra-laboratory variability.

Introduction

Alzheimer’s disease (AD) is the most prevalent age-related neurodegenerative disease, accounting for 60–80% of cases of dementia. The extracellular amyloid plaques arising from the accumulation of amyloid β42 protein (Aβ42) and intracellular neurofibrillary tangles formed by aggregations of hyperphosphorylated tau protein (P-tau) are the two main pathological hallmarks of AD (Serrano-Pozo et al., 2011). Both of these pathological characteristics are specific to AD, while neurodegeneration, characterized by an increase in total-tau protein (T-tau), is a non-specific biomarker that can be caused by several neurodegenerative diseases (Jack et al., 2018). Aβ42, P-tau, and T-tau are considered core AD biomarkers that can be measured in cerebrospinal fluid (CSF). Their use increases the accuracy of the diagnosis and prediction of the progression from mild cognitive impairment (MCI) to AD and can differentiate between AD and other causes of dementia or neuropsychiatric problems (Albert et al., 2011; McKhann et al., 2011; Sperling et al., 2011). In addition, the inclusion of these biomarkers in diagnosis benefits populations included in clinical trials (Jack et al., 2018).

Currently, enzyme-linked immunoassay (ELISA) is the most widely used approach for the detection of AD core biomarkers in CSF. However, these ELISA methods often show considerable inter and intra-lab variability that prevents the use of standard cut-off values and precludes the wide use of CSF biomarkers in clinical practice. To circumvent this problem, Fujirebio Diagnostics and Roche Diagnostics have recently developed fully automated platforms for the analysis of CSF biomarkers. Fujirebio has implemented four CSF analytes (Aβ42, Aβ40, T-tau, and P-tau) on the fully automated Lumipulse G System. The measurement method is based on a two-step sandwich chemiluminescent enzyme-immunoassay (CLEIA). The Lumipulse Aβ42 assay is standardized according to certified reference material (CRM) developed by the International Federation of Clinical Chemistry and Laboratory Medicine (IFCC) and the Joint Research Centre (JRC). These platforms consist of three CRMs based on human CSF, with low, middle and high concentrations of Aβ42. However, fully automated Elecsys assays for CSF Aβ42, T-tau and P-tau are run on Elecsys and Cobas e immunoassay analyzers (Roche Diagnostics GmbH, Penzberg, Germany). The measurement is performed based on the electrochemiluminescence immunoassay (ECLIA) in a two-step sandwich assay. The Elecsys Aβ42 assay has been standardized by a Joint Committee for Traceability in Laboratory Medicine (JCTLM) with an approved reference measurement procedure (RMP). Therefore, all assay lots are standardized to a sample set with target values derived from liquid chromatography–tandem mass spectrometry (LC-MS/MS) (Leinenbach et al., 2014).

Previous studies have evaluated the consistency between each of these automated methods with manual ELISA methods or Amyloid PET imaging (Janelidze et al., 2017; Hansson et al., 2018; Kollhoff et al., 2018; Schindler et al., 2018; Willemse et al., 2018; Alcolea et al., 2019; Bayart et al., 2019; Zecca et al., 2019; Kaplow et al., 2020). However, there are no studies that have compared the efficacy of Innotest, Lumipulse and Elecsys methods in a single cohort of patients.

The aims of this study were (a) to assess the concordance between core AD biomarkers measured in CSF using Innotest, Lumipulse and Elecsys methods; (b) to evaluate the diagnostic accuracy of biomarkers and their ratios measured by each method; (c) to assess the discriminating power of AT(N) groups that were generated by the results of the different biomarkers for each of these three technologies and (d) to define the CSF cut-off points for both Lumipulse and Elecsys assays based on the Lumipulse Aβ42/40 status.

Materials and Methods

Study Population

A total of 149 patients [AD (n = 34), MCI (n = 94) and non-AD dementias (n = 21)] were included in this study. The study population was recruited consecutively between July 2018 and July 2019 from patients attending the Cognitive Disorders Unit at the Hospital Universitari Santa Maria (Lleida, Spain). Inclusion criteria comprised presentation with suspected cognitive dysfunction at the memory clinic, for which the neurologist requested CSF analysis. The diagnosis of probable AD and MCI was performed based on NIAA criteria (Albert et al., 2011; McKhann et al., 2011). Each non-AD patient fulfilled the criteria for the specific diagnostic criteria of the disorder considered (e.g., Fronto-temporal dementia, Lewy body dementia, etc.) (Gorno-Tempini et al., 2011; Rascovsky et al., 2011; McKeith et al., 2017). The included patients signed an internal regulatory document stating that residual samples used for diagnostic procedures could be used for research studies without any additional informed consent.

CSF Collection and Storage

Cerebrospinal fluid samples were collected between 8 a.m. and 10 a.m. after an overnight fast into 10-mL polypropylene tubes (Sarstedt, 62.610.201). The tubes were inverted several times, and the CSF was processed based on the recommendations provided by each manufacturer. For the Lumipulse assay, the samples were centrifuged at 2,000 × g for 10 min at room temperature and aliquoted into two 2-mL polypropylene tubes (Sarstedt, 72.694.007), with each tube containing 1 mL of CSF. For the Elecsys method, the samples were aliquoted into two 0.5-mL polypropylene tubes (Sarstedt 72.730.005) after centrifugation. For the Innotest assay, the CSF was aliquoted into two 2-mL polypropylene tubes (Sarstedt, 72.694.007) after centrifugation. The samples were stored at −80°C until analyses.

CSF Analysis

Measurements of Aβ42 and Aβ40 (only for lumipulse), T-tau, and P-tau were performed at the clinical laboratory of Hospital Universitari Arnau de Vilanova, Lleida. On the day of the analysis, samples were thawed at room temperature, and the tubes were vortexed briefly. The biomarkers were measured directly from the storage tube and in five separate batches for all three methods. For each method, the same batch of reagents was used for each biomarker throughout the study, and for each sample, all analytes were quantified in the same run and from the same aliquot. For the ELISA method, Innotest Aβ42, Innotest htau-Ag, and Innotest P-tau (181P) assays (Fujirebio, Europe) were used. Innotest calibrator concentrations ranged from 63 to 4000 pg/mL for Aβ42, 40 to 2300 pg/mL for T-tau, and 16 to 1000 pg/mL for P-tau. According to previous analyses in clinical practice, cut-offs at our center were determined to be <600 pg/mL for Aβ42, >425 pg/mL for T-tau, and >65 pg/mL for P-tau. For the ECLIA method, the tubes analyzed using the Elecsys Aβ42 CSF, Elecsys T-tau CSF, and Elecsys P-tau (181P) CSF assays (Roche Diagnostics GmbH) were run on the cobas e 601 analyzer (Roche Diagnostics) per the manufacturer’s instructions. Elecsys measuring ranges were as follows: 200 to 1700 pg/mL for Aβ42, 80 to 1300 pg/mL for T-tau, and 8 to 120 pg/mL for P-tau. For data analysis, we used the cut-offs recommended by the manufacturer, which were as follows: ≤1000 pg/mL for Aβ42, >300 pg/mL for T-tau, and >27 pg/mL for P-tau. Seventeen samples had Aβ42 levels above the upper limit of the measuring range (1700 pg/mL) and were eliminated from the analysis. The results of the Elecsys Aβ42 assay were standardized to the JCTLM-approved RMP for quantitation of Aβ42 in human CSF, based on LC-MS/MS (Leinenbach et al., 2014). For the CLEIA technology, the CSF biomarkers were quantified using the Lumipulse Aβ42, Aβ40, T-tau, and P-tau (181P) assays on the LUMIPULSE G600II automated platform (Fujirebio) per the manufacturer’s instructions. Lumipulse measuring ranges were 9–2,335 pg/mL for Aβ42, 150–2,000 pg/mL for T-tau, and 1.1–400 pg/mL for p-Tau. The following cut-offs that had been determined by Fujirebio were used for data analysis: Aβ42 < 600 pg/mL, Aβ42/40 < 0.069, T-tau > 400 pg/mL, and P-tau > 56.5 pg/mL. The results of the Lumipulse Aβ42 presented in this work have been standardized with CRMs developed by the IFCC and JRC (Kuhlmann et al., 2017). The personnel involved in the CSF analyses were blind to the clinical diagnosis.

Statistical Analyses

All statistical analyses were performed using IBM SPSS version 25 (Armonk, NY, United States). One-way ANOVA and Chi-square tests were used for analysis of quantitative and qualitative variables, respectively. The quantitative variables were presented as means (±standard deviation, SD), and the qualitative variables were presented as percentages. To evaluate the correlation between methods, we used Pearson’s correlation coefficient (r), paired t-tests for paired samples, and the Bland-Altman plot. The diagnostic accuracy of the biomarkers/AT(N) classification was analyzed using a binary logistic regression model. In this model, the sensitivity was defined as the percentage of correct classification of AD diagnosis and the specificity as the percentage of correct classification of non-AD dementias diagnosis. Furthermore, we used this statistical model to evaluate the predictive value of the biomarkers with respect to AD prognosis. The receiver operating characteristic (ROC) analysis for evaluating diagnostic accuracy was further analyzed using the Hanley and McNeil method (Hanley and McNeil, 1982) to compare the Area Under the Curve (AUC). Values of |z| ≥ 1.96 were considered evidence that the true ROC areas were different. We also performed ROC analysis to determine the cut-offs for the core AD biomarkers and the ratios that best distinguished Lumipulse Aβ42/40+ individuals. In addition, the cut-offs were also determined based on the Innotest Aβ42 status. We determined the positive percent agreement (PPA) and negative percent agreement (NPA), and the single analyte value (or ratio) with the highest Youden index (PPA + NPA – 1) was identified as the cut-off value. Overall percent agreement (OPA) was defined as the sum of the Aβ42/40 + individuals who were positive for a CSF biomarker measure and the Aβ42/40 − individuals who were negative for a CSF biomarker measure divided by the cohort size, thereby providing an estimate of accuracy.

Results

Patient Characteristics

The demographic characteristics and biomarker results are summarized in Table 1. The average age of participants was 74 years, and 55% were female. Syndrome diagnoses in the cohort were the following: 34 (22.8%) with AD, 94 (63.1%) with MCI, and 21 (14.1%) with non-AD dementia. There were no significant differences between diagnostic groups for demographic and clinical variables except for MMSE score and hypertension (P < 0.0001 and P < 0.05, respectively). The mean MMSE score was lower [19.6 (4.2 SD)] for AD patients compared with the two other groups, followed by non-AD dementia patients [21.9 (4.6 SD)] and MCI subjects [25.2 (3.1 SD)]. For all three assays, all CSF biomarker concentrations were significantly different between the three diagnostic groups, except Lumipulse Aβ40 (P > 0.05). For Elecsys, samples that had Aβ42 values above the upper limit of detection (1700 pg/mL) were omitted from analysis (n = 17 samples, MCI 11, AD 4, and 2 non-AD patients).

TABLE 1

Table 1. The demographic characteristics and biomarker results for AD, MCI, and non-AD patients.

Concordance Between Innotest and Lumipulse Assays

Pearson’s correlations indicated a high correlation between biomarkers of both methods. Figure 1 shows the correlation and Bland-Altman plots for biomarkers quantified by Innotest and Lumipulse. The correlation coefficient between the two methods was 0.87 for Aβ42 (P < 0.0001), 0.95 for T-tau (P < 0.0001) and 0.95 for P-tau (P < 0.0001). The concordance between the values of the biomarkers between the two methods was assessed using paired sample t-tests. Our results indicated that there was high consistency in the Aβ42 (observed slope 0.98, t-test p = 0.319) and T-tau (observed slope 0.96, t-test P = 0.785) values between the two methods (Figure 1). Lumipulse Aβ42 values were slightly higher than those for Innotest, while Lumipulse T-tau values were slightly lower than those for Innotest; however, these differences were not statistically significant (Figure 1). As shown in the Bland-Altman plot, the bias (mean of the differences) for Aβ42 was 11.320 units (pg/mL) (continuous line) between the two methods. The regression line for the differences indicated that there was a non-significant negative trend in the differences as the magnitude of the measured variable increased. For T-tau, there was a bias of −2.157 units between the two methods. However, the results of Lumipulse and Innotest were not consistent with respect to P-tau values (observed slope 1.884, t-test P < 0.0001). For P-tau, there was a bias of −14.328 units (continuous line). The regression line for the differences indicated that there was a systematic proportional bias between the values of the two methods with a negative trend in the differences as the magnitude of P-tau values increased, especially for values greater than 50 pg/mL. Among all assays evaluated, approximately 95% of the measured values were within ±1.96 SD of the bias (Figure 1).

FIGURE 1

Figure 1. The correlation and Bland-Altman plots for Aβ42, T-tau, and P-tau measurements obtained by Lumipulse and Innotest ELISA methods (n = 149). Each point is defined as the measurements of Lumipulse and ELISA assays on the same biological sample. In correlation plots, the solid lines represent the estimated regression line, and the dotted line represents the identity line (x = y). In the Bland-Altman plots, solid lines represent the slope observed.

Concordance Between Innotest and Elecsys Assays

Pearson’s correlations indicated a high correlation between biomarkers of both methods. The correlation coefficient between the two methods was 0.88 for Aβ42 (P < 0.0001), 0.96 for T-tau (P < 0.0001) and 0.97 for P-tau (P < 0.0001). The paired samples t-test demonstrated that there was weak concordance between the two methods for all of the biomarkers. For all three biomarkers, the adjustment line (continuous line) (observed slope of 0.52 for Aβ42, t-test P < 0.0001; observed slope of 2.05 for T-tau, t-test P < 0.0001; and observed slope of 1.62 for P-tau, t-test P < 0.0001) was significantly separated from the perfect agreement line (dashed line) (Figure 2). The Bland-Altman plot indicated that there was a bias of −222.13 units (continuous line) between the two methods for Aβ42 (i.e., the Elecsys method quantified on average 222.13 pg/mL more Aβ42 than the Innotest assay). The regression line demonstrated a proportional systematic bias with a negative trend of differences as the magnitude of Aβ42 increased. For T-tau and P-tau, the biases (mean of differences) were 210.70 and 40.16 units, respectively. The regression line of the differences indicated a proportional systematic bias for both biomarkers with a positive trend of differences as the magnitude of these biomarkers increased. For all assays evaluated, approximately 95% of measured values were within ±1.96 SD of the bias (Figure 2).

FIGURE 2

Figure 2. The correlation and Bland-Altman plots for Aβ42, T-tau, and P-tau measurements obtained by Elecsys and Innotest methods (for P-tau and T-tau n = 145; for Aβ42 n = 135). Each point is defined as the measurement of Elecsys and Innotest assays on the same biological sample. In correlation plots, the solid lines represent the estimated regression line, and the dotted line represents the identity line (x = y). In the Bland-Altman plots, solid lines represent the slope observed.

Concordance Between Elecsys and Lumipulse Assays

There was a high correlation between all three biomarkers for both methods. The correlation coefficient between the two methods was 0.94 for Aβ42 (P < 0.0001), 0.95 for T-tau (P < 0.0001), and 0.96 for P-tau (P < 0.0001). Figure 3 shows the correlation and Bland-Altman plots for biomarkers quantified by Elecsys and Lumipulse. The t-test results indicated that there was weak concordance between all pairs of biomarkers (P < 0.0001). For all three biomarkers, the adjustment line (continuous line) (observed slope of 0.59 for Aβ42, t-test P < 0.0001; observed slope of 2.07 for T-tau, t-test P < 0.0001; and observed slope 3.21 for P-tau, t-test P < 0.0001) was significantly separated from the perfect agreement line (dashed line) (Figure 3). The Bland-Altman plot indicated that there was a bias of 243.28 (continuous line) for Aβ42, meaning that Lumipulse quantified 243.28 pg/mL less Aβ42 on average than Elecsys. The regression line demonstrated a proportional systemic bias with a positive trend of differences as the magnitude of Aβ42 increased. For T-tau and P-tau, the biases between the two assays were −210.754 and −54.128 units, respectively. The regression line of the differences indicated a proportional systematic bias for both biomarkers with a negative trend of differences as the magnitude of these biomarkers increased. For all assays evaluated, approximately 95% of measured values were within ± 1.96 SD of the bias (Figure 3).

FIGURE 3

Figure 3. The correlation and Bland-Altman plots for Aβ42, T-tau, and P-tau measurements obtained by Lumipulse and Elecsys methods (for P-tau and T-tau n = 145; for Aβ42 n = 137). Each point is defined as the measurement of Lumipulse and Elecsys assays on the same biological sample. In correlation plots, the solid lines represent the estimated regression line, and the dotted line represents the identity line (x = y). In the Bland-Altman plots, solid lines represent the slope observed.

AD Diagnostic Accuracy of the Biomarkers Quantified by Each Method

Using binary logistic regression, we evaluated the diagnostic accuracy of biomarkers quantified by each assay (clinical diagnosis is generally considered the gold standard). To discriminate AD from non-AD patients, the combined use of Aβ42 and P-tau was the best approach for all three assays. The Aβ42/40 ratio of Lumipulse also had high discriminating power, comparable with the combined use of Aβ42 and P-tau, to differentiate between the two diagnostic groups (AUC 0.882, 95% CI 0.785–0.980). Among all three methods, Lumipulse Aβ42 and P-tau had higher discriminating power with an AUC of 0.915 (95% CI 0.822–1.000). This combination of biomarkers had 91.2% sensitivity and 76.2% specificity for a correct classification of diagnostic groups, and their predictive accuracy was estimated to be 85.5%. However, the AUCs were not significantly different between the three methods, as they were assessed using the Hanley and McNeil method (|z| < 1.96). However, the sensitivity, specificity and predictive accuracy slightly differed between methods (Table 2).

TABLE 2

Table 2. Biomarkers with the best discriminating power between AD and non-AD dementia patients.

Diagnostic Accuracy of the AT(N) Classification for Each Method

The same statistical model was used to evaluate the discriminating power of the AT(N) classification for each method. We classified our study population into 6 AT(N) (0, 1, 2, 3, 4, and 5) groups based on the results of the three core AD biomarkers (Jack et al., 2018). Biomarkers were grouped into those for β amyloid deposition, pathologic tau, and neurodegeneration [AT(N)]. Here, A referred to levels of Aβ42 (Aβ42/40) in CSF, T referred to levels of P-tau in CSF, and (N) referred to levels of T-tau in CSF. We provided two AT(N) classifications for Lumipulse, one based on the results of Aβ42, T-tau and P-tau and the other one based on the Aβ42/40, T-tau and P-tau values (Table 3). Patients who were grouped as AT(N) 0 were negative for all three biomarkers. Patients in the AT(N) 1 group were only positive for Aβ42 or the Aβ42/40 ratio. AT(N) 2 patients were positive for Aβ42 or the Aβ42/40 ratio and P-tau. AT(N) 3 patients had positive results for all three biomarkers. AT(N) 4 patients were positive for Aβ42 or the Aβ42/40 ratio and T-tau. Finally, AT(N) 5 patients were negative for Aβ42 or the Aβ42/40 ratio but positive for P-tau or T-tau or both biomarkers. For Lumipulse and Elecsys assays, classification was made based on the cut-offs provided by the manufacturers. The cut-offs for the Innotest assay were determined in an independent cohort of patients and controls in our lab. Our results indicated that although AT(N) classification based on the Aβ42/40 had the best discriminating power to correctly separate AD patients from non-AD patients with dementia (AUC 0.798; 95% CI 0.649–0.947), there were no significant differences between the four AT(N) classifications [i.e., Innotest, 2 lumipulse and Elecsys biomarkers based on the AT(N) classifications] with respect to diagnostic accuracy after comparing AUCs with the Hanley and McNeil method (|z| < 1.96). However, the sensitivity, specificity, and total percentage of predictive accuracy were different between methods, especially between Lumipulse and Innotest or Elecsys (Table 3). Among the three methods, Lumipulse AT(N)s had the best sensitivity (91.2%) and total predictive accuracy, while Elecsys AT(N) had the best specificity (77.8%) for discriminating AD from non-AD dementia patients.

TABLE 3

Table 3. Diagnostic accuracy of the AT(N) classification for each method.

CSF Biomarker Cut-Offs Based on Aβ42/40 Ratio Status

As the Aβ42/40 ratio and AT(N) had the best diagnostic accuracy, we selected these variables to serve as references for determining the cut-offs of biomarkers and ratios for Lumipulse and Elecsys. The cut-offs for each biomarker or ratio were established to be values that optimized the concordance with Aβ42/40 status as positive/negative. The determined cut-offs in this study and the established cut-offs by Fujirebio and Roche Diagnostics are presented in Table 4. As displayed in Table 4, the cut-offs determined according to concordance with the Aβ42/40 ratio were comparable with the manufacturer cut-offs. For Lumipulse and Elecsys biomarkers and ratios, the AUC for discriminating Aβ42/40 status was greater than 0.8. To discriminate between Aβ42/40 positivity/negativity status among Lumipulse assays, the Aβ42/40 AUC was 100, as it was used for calculating the cut-offs. However, the P-tau/Aβ42 and T-tau/Aβ42 ratios had a high discriminating accuracy (AUC 0.922, OPA 95.9% and AUC 0.956, OPA 95.3%, respectively) at the cut-off values of ≥0.082 and ≥0.517, respectively. Among Elecsys markers, the P-tau/Aβ42 and T-tau/Aβ42 ratios had superior discriminating power (AUC 0.994, OPA 96.4% and AUC 0.974, OPA 95.6%, respectively) at the cut-off values of ≥0.023 and ≥0.26, respectively. In fact, the Elecsys P-tau/Aβ42 was the best in discriminating patients based on Aβ42/40 status (Table 4 and Figure 4).

TABLE 4

Table 4. Cut-offs of CSF biomarkers that yielded maximum Youden index versus Aβ42/Aβ40 ratio status in the receiver operating characteristics analysis.

FIGURE 4

Figure 4. CSF biomarkers that yielded the maximum Youden index versus Aβ42/Aβ40 ratio status in the receiver operating characteristics analysis.

In addition, we determined the cut-offs for biomarkers and ratios of Lumipulse and Elecsys based on the Aβ42 status of Innotest (Supplementary Table 1). Among Lumipulse assays, Aβ42 had the highest AUC (0.955, 95% CI 0.924–0.986) followed by P-tau/Aβ42 (AUC 0.933, 95% CI 0.891–0.976) and Aβ42/40 (AUC 0.931, 95% CI 0.892–0.970). For the Elecsys assay, Aβ42 had the highest AUC (0.974, 95% CI 0.954–0.995), followed by P-tau/Aβ42 (AUC 0.936, 95% CI 0.894–0.978) and T-tau/Aβ42 (AUC 0.920, 95% CI 0.868–0.971).

Furthermore, all statistical analysis was performed for the study cohort separated by sex (Supplementary Figures 1–8 and Supplementary Tables 2–7). We found a high correlation between the three methods for both male and female subjects, although there were systematic differences between biomarker values measured by each method (Supplementary Figures 1–6). Interestingly, the cut-offs of CSF biomarkers for male subjects was lower than those of manufacturer, while in the case of female subjects these cut-offs were comparable with corresponding manufacturer’s cut-offs (Supplementary Tables 6, 7).

Discussion

In this study, we evaluated the concordance between three different methods for measurement of AD CSF biomarkers—Innotest ELISA, Elecsys and Lumipulse platforms—in a cohort of patients with AD, MCI, and non-AD dementias. We also evaluated the diagnostic accuracy of biomarkers and their ratios measured by each method. Furthermore, we determined cut-offs for CSF biomarkers of AD (Aβ42, T-tau, and P-tau) and their ratios measured on the fully automated Lumipulse and Elecsys to optimize their concordance with Aβ42/40 status.

Although there was a high correlation between all three assays, our results showed that there was a lack of consistency between the three methods, except for Aβ42 and T-tau of Lumipulse and Innotest. Because the antibodies used for the Lumipulse assays were produced by the same manufacturer as Innotest ELISA (Fujirebio), the similar specificity of the antibodies between the two methods may partly explain the concordance we observed between Aβ42 and T-tau values between these two methods. Our results were consistent with previous studies that had found a systematic bias between the measurements of biomarkers by Lumipulse and Innotest (Bayart et al., 2019) and by Elecsys and Innotest (Willemse et al., 2018). The lack of concordance between Elecsys, Lumipulse, and Innotest assays may be attributed to the differences that exist between these methods. First, they have different recommended pre-analytical procedures that can affect the measured concentration of CSF biomarkers. Among these three biomarkers, Aβ42 is known to be more sensitive to pre-analytical conditions. Second, these methods use different measurement technologies (ECLIA, CLEIA, and ELISA, respectively), which may affect the detectable concentration. Third, the antibodies that were produced and applied in the AD CSF assays by Roche Diagnostics and Fujirebio Diagnostics may have different specificities. Finally, although both Elecsys and Lumipulse have been standardized for Aβ42, the material used for standardization differed between methods (Bittner et al., 2016; Kuhlmann et al., 2017).

We examined the ability of Aβ42, T-tau, P-tau and their ratios to discriminate AD patients from patients with non-AD dementias. Aβ42 and P-tau combined were both the best biomarkers for discriminating between the two diagnostic groups. Both biomarkers were specific to AD; therefore, it was not surprising that their combination had a high discriminating power for diagnosing AD patients. However, abnormal concentrations of T-tau in CSF, which underlies neurodegeneration, is not specific to AD and occurs in non-AD dementias or in non-AD elderly persons with comorbidities (Kovacs et al., 2013). Evaluation of differences in AUC revealed that there were no significant differences in the discriminating power of Aβ42 + P-tau measured by each method. However, the Innotest Aβ42 + P-tau had a better sensitivity (94%), specificity (76%), and predictive accuracy (87%). The Aβ42/40 ratio also had a high discriminating power for differentiating between patients with AD and non-AD dementias. Consistent with our results, Shoji et al. (1998) and Lewczuk et al. (2004) previously suggested that the Aβ42/40 ratio is superior to the concentration of Aβ42 alone for discriminating AD patients.

We also assessed the discriminating power of AT(N) groups that were generated by the results of CSF Aβ42 (Aβ42/40), P-tau and T-tau for each method. The AT(N) classification was proposed by the NIAA research framework (Jack et al., 2018) and gives a biological rather than a clinical definition of AD. We found that the use of Aβ42/40 instead of Aβ42 in AT(N) improved the classification accuracy (AUC 0.798, 95% CI 0.649–0.947 vs. AUC 0.778, 95% CI 0.617–0.939), However, the sensitivity, specificity and predictive accuracy was the same for both AT(N)s. Among all four AT(N) classifications, Elecsys AT(N) had the highest specificity. In fact, Elecsys AT(N) had better specificity than sensitivity in discriminating the two diagnostic groups. The preference for higher sensitivity or specificity depends on the purpose of different investigation scenarios. For example, for screening purposes, higher sensitivity is always preferable; however, high specificity might be preferable for the selection of patients for clinical trials. These results should be interpreted with caution because of the small population size of both of the diagnostic groups in our study.

Finally, we defined the CSF cut-offs for both Lumipulse and Elecsys assays based on the Lumipulse Aβ42/40 status because of its high diagnostic accuracy in our study, its high stability with respect to pre-analytical variations (Lewczuk et al., 2006; Willemse et al., 2018) and the fact that the ratio probably accounts for inter-individual variability in overall Aβ production and CSF turnover (Janelidze et al., 2017). Given that Innotest assays are among some of the most commonly used methods for the detection of AD CSF biomarkers, we also provided the cut-offs for both Lumipulse and Elecsys assays based on the Innotest Aβ42 status.

In previous studies, amyloid PET visual read (Schindler et al., 2018; Alcolea et al., 2019) or diagnostic accuracy (Bayart et al., 2019) have been used for the determination of AD CSF biomarkers cut-offs for fully automated methods and their ratios. Our results indicated that the cut-offs based on the Aβ42/40 ratio had a close similarity to the cut-offs established by each manufacturer; therefore, the Aβ42/40 ratio is a robust variable that can differentiate AD from non-AD individuals. Based on our results for both the Lumipulse and the Elecsys methods, P-tau/Aβ42 and T-tau/Aβ42 performed better together than each biomarker alone in discriminating Aβ42/Aβ40 ± status. This result is consistent with the results of previous studies where P-tau/Aβ42 (Alcolea et al., 2019) or T-tau/Aβ42 (Bayart et al., 2019) demonstrated superior performance in discriminating the diagnostic groups or amyloid PET status compared with individual biomarkers (Schindler et al., 2018).

Some limitations of this study require consideration. First, our study population lacked health control individuals. The majority of the population consisted of MCI subjects (n = 94) with a short follow-up time; for this reason, we decided to eliminate patients in some analyses and retain a small number of AD (n = 34) and non-AD demented patients (n = 21) when evaluating the diagnostic accuracy of biomarkers. Second, instead of using an independent method, we used the Aβ42/Aβ40 ratio status or Aβ42 status to determine the biomarker cut-offs, and this may have led to the overfitting of the results. Third, Aβ40 cannot be measured by Elecsys or ELISA, so, the comparison was incomplete. Other limitation is that 17 patients were excluded of the analyses because they had Aβ42 values above the upper limit of detection (1700 pg/ml) for Elecsys.

The main strength of our study is that we compared, for the first time, the clinical and analytical performance of fully automated Elecsys and Lumipulse platforms together in the same cohort of patients. In addition, our study population consisted of a real population of patients who attended a memory clinic and, therefore, provided a more realistic application of biomarkers in daily clinical practice.

Conclusion

In conclusion, both Lumipulse and Elecsys methods had a high correlation with each other and with Innotest ELISA. The presence of systematic bias between biomarkers measured by each method was expected as there were various pre-analytical and analytical differences between methods. For both Lumipulse and Elecsys methods, ratios had a better analytical performance compared with individual biomarkers, and the Aβ42/Aβ40 ratio had a high concordance with the diagnostic accuracy of AD. Because the calibrators were adjusted with reference samples in both automated platforms, it was expected that these platforms would reduce intra- and inter-laboratory variations and enhance reproducibility.

Author’s Note

Considering the importance of study of cerebrospinal fluid biomarkers in mild cognitive impairment and Alzheimer’s disease, we aimed to investigate the concordance between core AD biomarkers measured in CSF using Innotest, Lumipulse and Elecsys methods. We observed that both, Lumipulse and Elecsys methods had a high correlation with each other and with Innotest ELISA. The presence of systematic bias between biomarkers measured by each method was expected as there are various pre-analytical and analytical differences between methods. For both Lumipulse and Elecsys methods, ratios had a better analytical performance compared with individual biomarkers. The Lumipulse and Elecsys CSF AD assays showed high analytical and clinical performances so their use is recommended for the measurement of CSF AD biomarkers compared with unstandardized manual methods.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by the Comite Etica Hospital Arnau Vilanova Lleida. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

FD, RL-O, and GP-R designed the study, analyzed the data, interpreted the data, and wrote the manuscript. IR-L, AA, RH, NT, and MR-J collected the data. All authors revised the manuscript and approved it for submission.

Funding

This study was supported by the Generalitat of Catalonia, Department of Health (PERIS 2019 SLT008/18/00050 to GP-R). IRBLleida is a CERCA Program/Generalitat of Catalonia. FD was supported by Agency for Management of University and Research Grants and European Social Fund (FI_B100153).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnagi.2021.604119/full#supplementary-material

References

Albert, M. S., DeKosky, S. T., Dickson, D., Dubois, B., Feldman, H. H., Fox, N. C., et al. (2011). The diagnosis of mild cognitive impairment due to Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement. 7, 270–279.

Google Scholar

Alcolea, D., Pegueroles, J., Muñoz, L., Camacho, V., López-Mora, D., Fernández-León, A., et al. (2019). Agreement of amyloid PET and CSF biomarkers for Alzheimer’s disease on lumipulse. Ann. Clin. Transl. Neurol. 6, 1815–1824. doi: 10.1002/acn3.50873

PubMed Abstract | CrossRef Full Text | Google Scholar

Bayart, J. L., Hanseeuw, B., Ivanoiu, A., and van Pesch, V. (2019). Analytical and clinical performances of the automated Lumipulse cerebrospinal fluid Aβ42 and T-Tau assays for Alzheimer’s disease diagnosis. J. Neurol. 266, 2304–2311. doi: 10.1007/s00415-019-09418-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Bittner, T., Zetterberg, H., Teunissen, C. E., Ostlund, R. E. Jr., Militello, M., Andreasson, U., et al. (2016). Technical performance of a novel, fully automated electrochemiluminescence immunoassay for the quantitation of β-amyloid (1-42) in human cerebrospinal fluid. Alzheimers Dement. 12, 517–526. doi: 10.1016/j.jalz.2015.09.009

PubMed Abstract | CrossRef Full Text | Google Scholar

Gorno-Tempini, M. L., Hillis, A. E., Weintraub, S., Kertesz, A., Mendez, M., Cappa, S. F., et al. (2011). Classification of primary progressive aphasia and its variants. Neurology 76, 1006–1014.

Google Scholar

Hanley, J., and McNeil, B. (1982). The meaning and use of the area under a Receiver Operatin Characteristic (ROC) curve. Radiology 143, 29–36. doi: 10.1148/radiology.143.1.7063747

PubMed Abstract | CrossRef Full Text | Google Scholar

Hansson, O., Seibylc, J., Stomrud, E., Zetterberg, H., Trojanowskih, J., Bittneri, T., et al. (2018). CSF biomarkers of Alzheimer’s disease concord with amyloid-β PET and predict clinical progression: a study of fully automated immunoassays in BioFINDER and ADNI cohorts. Alzheimers Dement. 14, 1470–1481. doi: 10.1016/j.jalz.2018.01.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Jack, C. Jr., Bennett, D., Blennow, K., Carrillo, M., Dunn, B., Haebeerlein, S., et al. (2018). NIA-AA research framework: toward a biological definition of Alzheimer’s disease. Alzheimers Dement. 14, 535–562. doi: 10.1016/j.jalz.2018.02.018

PubMed Abstract | CrossRef Full Text | Google Scholar

Janelidze, S., Pannee, J., Mikulskis, A., Chiao, P., Zetterberg, H., Blennow, K., et al. (2017). Concordance between different amyloid immunoassays and visual amyloid positron emission tomographic assessment. JAMA Neurol. 74, 1492–1501. doi: 10.1001/jamaneurol.2017.2814

PubMed Abstract | CrossRef Full Text | Google Scholar

Kaplow, J., Vandijck, M., Gray, J., Kanekiyo, M., Huyck, E., Traynham, C. J., et al. (2020). Concordance of Lumipulse cerebrospinal fluid t-tau/Aβ42 ratio with amyloid PET status. Alzheimers Dement. 16, 144–152. doi: 10.1002/alz.12000

PubMed Abstract | CrossRef Full Text | Google Scholar

Kollhoff, A. L., Howell, J. C., and Hu, W. T. (2018). Automation vs. experience: measuring Alzheimer’s beta-amyloid 1-42 peptide in the CSF. Front. Aging Neurosci. 10:253. doi: 10.3389/fnagi.2018.00253

PubMed Abstract | CrossRef Full Text | Google Scholar

Kovacs, G. G., Milenkovic, I., Wohrer, A., Hoftberger, R., Gelpi, E., Haberler, C., et al. (2013). Non-Alzheimer neurodegenerative pathologies and their combinations are more frequent than commonly believed in the elderly brain: a community-based autopsy series. Acta Neuropathol. 126, 365–384. doi: 10.1007/s00401-013-1157-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Kuhlmann, J., Andreasson, U., Pannee, J., Bjerke, M., Portelius, E., Leinenbach, A., et al. (2017). CSF Ab 1–42 –an excellent but complicated Alzheimer’s biomarker – a route to standardisation. Clin. Chim. Acta 467, 27–33. doi: 10.1016/j.cca.2016.05.014

PubMed Abstract | CrossRef Full Text | Google Scholar

Leinenbach, A., Pannee, J., Dulffer, T., Huber, A., Bittner, T., Andreasson, U., et al. (2014). Mass spectrometry-based candidate reference measurement procedure for quantification of amyloid-beta in cerebrospinal fluid. Clin. Chem. 60, 987–994. doi: 10.1373/clinchem.2013.220392

PubMed Abstract | CrossRef Full Text | Google Scholar

Lewczuk, P., Beck, G., Esselmann, H., Bruckmoser, R., Zimmermann, R., Fiszer, M., et al. (2006). Effect of sample collection tubes on cerebrospinal fluid concentrations of tau proteins and amyloid beta peptides. Clin. Chem. 52, 332–334. doi: 10.1373/clinchem.2005.058776

PubMed Abstract | CrossRef Full Text | Google Scholar

Lewczuk, P., Esselmann, H., Otto, M., Maler, J. M., Henkel, A. W., Henkel, M. K., et al. (2004). Neurochemical diagnosis of Alzheimer’s dementia by CSF Abeta42, Abeta42/Abeta40 ratio and total tau. Neurobiol. Aging 25, 273–281.

Google Scholar

McKeith, I. G., Boeve, B. F., Dickson, D. W., Halliday, G., Taylor, J. P., Weintraub, D., et al. (2017). Diagnosis and management of dementia with Lewy bodies: fourth consensus report of the DLB consortium. Neurology 89, 88–100.

Google Scholar

McKhann, G. M., Knopman, D. S., Chertkow, H., Hyman, B. T., Jack, C. R. Jr., Kawas, C. H., et al. (2011). The diagnosis of dementia due to Alzheimer’s disease: recommendations from the National Institute on Aging Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement. 7, 263–269.

Google Scholar

Rascovsky, K., Hodges, J. R., Knopman, D., Mendez, M., Kramer, J. H., van Swieten, J., et al. (2011). Sensitivity of revised diagnostic criteria for the behavioural variant of frontotemporal dementia. Brain 134, 2456–2477.

Google Scholar

Schindler, S., Graya, J., Gordona, B., Xionga, C., Batrla-Utermannf, R., Quan, M., et al. (2018). Cerebrospinal fluid biomarkers measured by Elecsys§assays compared to amyloid imaging. Alzheimers Dement. 14, 1460–1469. doi: 10.1016/j.jalz.2018.01.013

PubMed Abstract | CrossRef Full Text | Google Scholar

Serrano-Pozo, A., Frosch, M. P., Masliah, E., and Hyman, B. T. (2011). Neuropathological alterations in Alzheimer disease. Cold Spring Harb. Perspect. Med. 1:a006189. doi: 10.1101/cshperspect.a006189

PubMed Abstract | CrossRef Full Text | Google Scholar

Shoji, M., Matsubara, E., Kanai, M., Watanabe, M., Nakamura, T., Tomidokoro, Y., et al. (1998). Combination assay of CSF tau, A beta 1-40 and A beta 1-42(43) as a biochemical marker of Alzheimer’s disease. J. Neurol. Sci. 158, 134–140. doi: 10.1159/000078534

CrossRef Full Text | Google Scholar

Sperling, R. A., Aisen, P. S., Beckett, L. A., Bennett, D. A., Craft, S., Fagan, A. M., et al. (2011). Toward defining the preclinical stages of Alzheimer’s disease: recommendations from the National Institute on Aging Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement. 7, 280–292.

Google Scholar

Willemse, E., van Maurikb, I., Tijmsb, B., Bouwmanb, F., Franked, A., Hubeeke, I., et al. (2018). Diagnostic performance of Elecsys immunoassays for cerebrospinal fluid Alzheimer’s disease biomarkers in a nonacademic, multicenter memory clinic cohort: the ABIDE project. Alzheimers Dement. 10, 563–572. doi: 10.1016/j.dadm.2018.08.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Zecca, C., Brescia, V., Piccininni, M., Capozzo, R., Barone, R., Barulli, M. R., et al. (2019). Comparative evaluation of two immunoassays for cerebrospinal fluid β-Amyloid1–42 measurement. Clin. Chim. Acta 493, 107–111. doi: 10.1016/j.cca.2019.02.033

PubMed Abstract | CrossRef Full Text | Google Scholar