Validation of Predictive Models for Autoimmune Encephalitis-Related Antibodies to Cell-Surface Proteins Expressed in Neurons: A Retrospective Study Based in a Hospital

Objective: Autoimmune encephalitis (AE) is a severe but treatable autoimmune disorder that is diagnosed by antibody (Ab) testing. However, it is unrealistic to obtain an early diagnosis in some areas since the Ab status cannot be immediately determined due to time and technology restrictions. In our study, we aimed to validate the Antibody Prevalence in Epilepsy and Encephalopathy (APE²) score among patients diagnosed with possible AE as a predictive model to screen AE patients with antibodies to cell-surface proteins expressed in neurons.

Methods: A total of 180 inpatients were recruited, and antibodies were detected through serological and/or cerebrospinal fluid (CSF) evaluations. The APE² score was used to validate the predictive models of AE with autoantibodies.

Results: The mean APE² score in the Ab-positive cases was 7.25, whereas the mean APE² score in the Ab-negative cases was 3.18 (P < 0.001). The APE² score had a receiver operating characteristic (ROC) area under the curve of 0.924 [P < 0.0001, 95% confidence interval (CI) = 0.875–0.973]. With a cutoff score of 5, the APE² score had the best psychometric properties, with a sensitivity of 0.875 and a specificity of 0.791.

Conclusion: The APE² score is a predictive model for AE with autoantibodies to cell-surface proteins expressed in neurons and was validated and shown to have high sensitivity and specificity in our study. We suggest that such a model should be used in patients with a suspected diagnosis of AE, which could increase the detection rate of Abs, reduce testing costs, and help patients to benefit from treatment quickly.

Introduction

Autoimmune encephalitis (AE) is an immune-mediated neurological disorder characterized by rapidly progressive central nervous system (CNS) symptoms that is associated with specific autoantibodies targeting cell-surface neuronal antigens (1). AE is classified into different subtypes according to the neuronal antigens targeted by the autoantibodies found in the serum and/or cerebrospinal fluid (CSF) of patients, each of which has different clinical characteristics and outcomes (1, 2). The identified forms of AE might be associated with antibodies (Abs) against intracellular antigens, synaptic receptors, ion channels, or cell-surface proteins, according to the location of these specific autoantibodies (3).

As more related cases have been identified in the past 10 years, there has been increasing interest in the pathogenesis and clinical features of AE, especially in patients with Abs to cell-surface proteins expressed in neurons, including antibodies against synaptic receptors and antibodies targeting ion channels and cell-surface proteins, who have an effective response to immunosuppressive therapies and who respond well to immunosuppressive therapies (4). Abs to N-methyl-D-aspartate receptor (NMDAR), gamma-aminobutyric acid A receptor (GABAAR), gamma-aminobutyric acid B receptor (GABABR), alpha-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid receptor (AMPAR), metabotropic glutamate receptor 5 (mGluR5), dopamine 2 receptor, leucine-rich glioma-inactivated 1 (LGI1), contactin-associated protein-like 2 (Caspr2), and dipeptidyl-peptidase-like protein 6 (DPPX) were identified in patients and were associated with various clinical manifestations (5).

The current diagnostic criteria for AE depend heavily on Ab testing and responses to immunotherapy (6). However, the identification of the different forms of Abs involved with AE is a complicated process. The diagnosis of AE should be suspected based on Ab identification, as well as combination of disease signs and symptoms (3). In addition, some patients with AE may not respond to immunotherapy or may require intensive and long-term treatment, which is unavailable in most health care systems unless a diagnosis has been previously established (2). However, rapid access to gold standard diagnostic cell-based assays is not universally available, especially in some areas where medical conditions are limited. An Ab prediction model that is not based on Ab detection not only is helpful for early diagnosis but also can save medical resources and reduce the economic burden of patients. These issues prompted researchers to explore methods for the early diagnosis of AE and to establish predictive models for the detection of autoantibodies based on the clinical presentation and initial neurologic evaluations prior to Ab testing.

Recently, an Antibody Prevalence in Epilepsy and Encephalopathy (APE²) score was described as a model for predicting the detection of neural-specific autoantibodies based on clinical characteristics; this model was validated and had high sensitivity and specificity for identifying patients with cognitive decline (7–9). AE usually presents with the subacute onset of memory deficits or an altered mental status, which may or may not be accompanied by other symptoms (2). Therefore, we aimed to validate the APE² scoring system as a predictive model to screen AE patients with antibodies to cell-surface proteins expressed in neurons among patients diagnosed with possible AE.

Methods

Subjects

We conducted a retrospective review of inpatients diagnosed with possible AE between June 2014 and June 2019 at the First Affiliated Hospital of Wenzhou Medical University, China. The levels of AE autoantibodies of every inpatient were detected through serological and/or CSF evaluations. Patients were included in the study based on the following criteria: (1) onset of working memory deficits (short-term memory loss), altered mental status, or psychiatric symptoms; (2) at least one of the following: new focal CNS findings, seizures not explained by a previously known seizure disorder, CSF pleocytosis [white blood cell (WBC) count of more than 5 cells per mm], and magnetic resonance imaging (MRI) features suggestive of encephalitis; and (3) reasonable exclusion of alternative causes. Patients with incomplete medical record information were excluded.

Ab Evaluations

All serum and CSF specimens were screened by standardized indirect immunofluorescence assays (IFAs) and cell-based assays (CBAs) using human embryonic kidney (HEK) 293 cells transfected with appropriate expression plasmids to confirm IgGs specific for NMDAR, AMPA1, AMPA2, LGI1, CASPR2, GABABR, and DPPX. The initial dilution titers of CSF and serum were 1:1 and 1:10, respectively.

Data Collection

Data were gathered from each patient, which included basic demographic information (age and sex), the duration of symptoms before hospitalization, the presence and type of seizures, status epilepticus, mental disorders, sleep disorders, headache, laboratory data, and CSF results (protein, WBCs, glucose, and chloride), MRI, anticardiolipin antibody (ACA), antinuclear antibody (ANA), treatment, intensive care unit (ICU) admission, the need for mechanical ventilation, and the prognosis during hospitalization. The APE² score of each inpatient was evaluated by a neurologist based on the clinical manifestations, and the neurologists were blinded to the actual results of Ab evaluations.

APE² Score

The initial version of the Antibody Prevalence in Epilepsy (APE) score was established by Dubey as a model to predict the detection of neural Abs in autoimmune epilepsy patients based on clinical manifestations and initial neurologic evaluations (7). An APE score of ≥4 had a sensitivity and specificity of 82.6 and 82.0%, respectively, and can be used as a tool for identifying patients for Ab testing (7). After three modifications, the APE² scoring system has a higher specificity for the prediction of neural autoantibody positivity (from 78 to 84%) among patients with epilepsy, without a loss in sensitivity (98%) (9). The determination of the APE² score in patients with encephalopathy or cognitive decline to predict AE-related Ab positivity showed that an APE² score ≥4 was 99% sensitive and 93% specific for neural-specific Abs (8).

Statistical Analysis

The data were analyzed using the Statistical Package for the Social Sciences (SPSS version 20.0; SPSS Inc., Chicago, IL, USA). Sociodemographic characteristics and clinical variables are presented as counts, frequency (%), means, and standard deviations. Categorical variables were compared using chi-square tests or Fisher's exact tests. Interval variables were performed using Mann–Whitney U-tests. The threshold for statistical significance was set at P < 0.05 (two-tailed). The odds ratio (OR) and 95% confidence intervals (CIs) were used to quantify the strength of the associations. Receiver operating characteristic (ROC) curve analysis was performed to assess the sensitivity, specificity, Youden index, and positive and negative predictive values of the APE² score at different cutoff scores.

Results

Serum, CSF, or both were collected from 238 patients at the onset of the study for the detection of AE-related Ab testing. Thirty-five patients were excluded due to an indefinite diagnosis of encephalitis, whereas 23 patients were excluded due to incomplete records and insufficient clinical data. The final study group consisted of 180 inpatients, among whom 102 patients underwent simultaneous serum and CSF examinations, 38 patients underwent only serum examinations, and 40 patients underwent only CSF examinations. AE-related antibodies in the serum/CSF were detected in 32 of the 180 inpatients, including NMDAR (n = 23), LGI1 (n = 3), GABABR (n = 3), CASPR2 (n = 2), and AMPA1 (n = 1), which were in accordance with the diagnostic standards of AE. A large number of patients with AE have no well-defined syndrome. According to three types of specific syndromes associated with AE (5), there were anti-NMDAR encephalitis (n = 23), limbic encephalitis (n = 7), and Morvan's syndrome (n = 2). Among these 32 patients, 28 (87.5%) had new-onset seizures, and 28 (87.5%) had mental status changes. Regarding prognosis, 23 (71.9%) patients recovered well, 7 (21.9%) patients did not respond to treatment, and 2 (6.3%) patients were discharged after the deterioration of their condition. Table 1 shows the detailed information of the Ab-positive group and the Ab-negative group, such as the basic demographic information, clinical features, laboratory data, treatment, and prognosis.

TABLE 1

Table 1. Clinical characteristics of patients in the study.

In Table 2, we compared the differences in the APE² score between the two groups and evaluated the associations with clinical characteristics. According to the results of univariate analysis, the following items were associated with the Ab-positive group: neuropsychiatric changes (P < 0.001, OR = 5.930), autonomic dysfunction (P < 0.001, OR = 11.308), viral prodrome (P < 0.001, OR = 3.655), faciobrachial dystonic seizures (P < 0.001, OR = 6.692), facial dyskinesias (P = 0.016, OR = 2.964), refractory epilepsy (P < 0.001, OR = 7.367), and encephalitis and a cancer diagnosis within 5 years of encephalopathy or cognitive dysfunction (P = 0.018, OR = 4.552). Conversely, there were no associations between Ab positivity and subacute encephalopathy/new-onset epilepsy (P = 0.113), CSF inflammatory changes (P = 0.564), or MRI changes (P = 0.702). The mean APE² scores (Table 1) were 7.25 in the Ab-positive group and 3.18 in the Ab-negative group (P < 0.001).

TABLE 2

Table 2. Components of the APE² score.

The area under the ROC of the APE² score was 0.924 (P < 0.0001, 95% CI = 0.875–0.973) (Figure 1). As a model to predict Ab-positive AE, an APE² score ≥5 had a sensitivity of 0.875, a specificity of 0.791, and a Youden index of 0.666 (Table 3). The Hosmer–Lemeshow test P-value was 0.827 in our study.

FIGURE 1

Figure 1. ROC analyses of the APE² score. ROC, receiver operating characteristic.

TABLE 3

Table 3. ROC and diagnostic efficiency of the APE² score for the diagnosis of AE with autoantibodies to cell-surface proteins expressed in neurons.

We also examined the score distribution of each patient in Figure 2 to perform further comparisons. NMDAR Ab cases were compared separately from other Ab cases (such as LGI1, GABABR, CASPR2, and AMPA1) in the 32 patients with positive Abs, and the mean APE² scores were 7.74 and 6, respectively (P = 0.058).

FIGURE 2

Figure 2. The scores distribution of APE² in each patient. NMDAR antibody cases: n = 23; other antibody-positive cases including LGI1 (n = 3), GABABR (n = 3), CASPR2 (n = 2), and AMPA1 (n = 1).

Discussion

This retrospective study was based in a hospital. Among the patients diagnosed with possible AE, 17.8% had AE-related Abs, which were not present at a low rate. This study suggests that AE among hospitalized patients with encephalopathy is not well-recognized. As a prediction model for AE, the APE² scoring system has high sensitivity and specificity, which makes this prediction model more widely used.

The clinical manifestations of AE varied. A large number of patients with AE do not present with a defined syndrome (10). Autoantibody testing is the most clear and important way to determine the diagnosis of these different conditions, and we can define comorbidities, tumor associations, and prognosis according to these testing results (2). However, immune-mediated limbic encephalitis can occur without detectable autoantibodies, which means that the absence of autoantibodies does not exclude the possibility of AE. In our study, a patient with abnormal behavior and apathy as initial symptoms responded well to immunotherapy. However, the previous two autoimmunity antibodies tested in the CSF were negative, and NMDAR antibodies were found in the third test. Hence, more methods are needed to screen AE patients with antibodies to cell-surface proteins expressed in neurons in clinical practice. As a predictive model with high sensitivity and specificity, the APE² score is well-defined, easily measured, routinely available, and can be used at the bedside. Therefore, the use of the APE² score during diagnosis to screen out suspicious patients is beneficial.

The inclusion criteria used in our study were based on Graus's criteria. However, the duration of symptoms before hospitalization was not strictly defined. A subacute disease duration (rapid progression of <3 months) is important to the diagnosis of possible AE. When we reviewed the patients' histories, we found that some patients first presented with a single neurological or psychiatric symptom and developed other symptoms weeks or even months after the onset, which may have led to delays in seeking medical attention. Patients whose disease duration was longer than 3 months but met the other inclusion criteria described above were still included in the study to avoid missing some possible patients with clinical presentations similar to AE.

As shown in our study, among 148 patients without autoantibodies in the serum/CSF, 31 (20.9%) patients had APE² scores ≥5. We cannot exclude the possibility that those patients diagnosed with AE had antibodies against intracellular antigens or synaptic receptors. AE can be caused by the production of several different autoantibodies targeting various neuronal antigens (2). Our study focused on AE with antibodies to cell-surface proteins expressed in neurons for several reasons. First, antibodies against intracellular antigens are defined as paraneoplastic antibodies because they are frequently observed in patients with cancer (11). The most important antibodies included in this group are against Ma2, Hu, and glutamic acid decarboxylase (GAD). AE patients with these antibodies respond significantly worse to immunosuppressive therapies than other AE patients (12). We aimed to predict the presence of antibodies to cell-surface proteins expressed in neurons based on the APE² score and to alleviate the clinical symptoms of patients through timely immunotherapy without the overutilization of these resources, which is very important in the hospital testing. On the other hand, in some areas with limited medical conditions, it is difficult to obtain early diagnosis by determining the status of antibodies. An Ab prediction model that is not based on Ab detection is more practical and convenient for such an applicable range. Moreover, known antibodies might only be the tip of the iceberg, and with advances in detection technology, potential new antibodies targeting the structure of neurons have been identified at an increasing rate.

It should be noted that some AE-related antibodies might be found only in the CSF (13, 14). An observational study showed that antibodies were found in the CSF among 250 patients with anti-NMDAR encephalitis, whereas researchers could find hardly any antibodies in the serum of 14% (36) of these patients (15). Furthermore, when the Abs found were different in the CSF and serum in one patient, although it did not occur in our study, the types of Abs in the CSF were usually determined (16). These findings supported the higher sensitivity of Ab testing in the CSF than in the serum. In our study, 102 patients out of 180 underwent both serum and CSF evaluations. Although 23% of Ab-negative patients did not undergo autoantibody testing in the CSF, which could have led to false-negative results, both CSF and serum were used for neuronal Ab testing in patients in the Ab-positive group. More importantly, among 32 cases, only 2 patients had low Ab titers [NMDAR Ab, 1:10 (all CSF)]; the remaining 30 patients had high titers [range 1:16–1:1,000 (serum or CSF)]. Although the univariate analysis results found no associations between patients with only serum positivity and those with serum and/or CSF positivity (P = 0.946), we still recommend that both CSF and serum be used for autoantibody testing in patients with suspected AE.

Statistical analysis showed no significant differences in some items of the APE² score. Firstly, the manifestations of the inpatients included in our study were almost mental disorders or seizures. With the exception of 24 patients with symptoms of encephalopathy for more than 1–6 weeks, the remaining 156 (86.7%) patients scored 1 point on the first item. There were also no statistically significant differences in the items related to CSF inflammatory changes (P = 0.564) and MRI changes (P = 0.702). A potential reason is that Ab-negative cases included seronegative AE patients. Furthermore, 25.7% of patients only checked the AE-related antibodies to cell-surface proteins expressed in neurons in the serum for this group. Therefore, patients who met the diagnostic criteria for possible AE should be investigated separately. It is possible that the APE² scores might help us identify seronegative AE patients. The weighted points of these three items in the APE² score system are 1, 2, and 2, respectively, which affects the overall scores as well as the sensitivity and specificity of the APE² score.

Our research was based on a relatively small sample. Nevertheless, the small sample did not significantly affect the precision of the study estimates because the incidence rate of AE is rather low. As one of the most common forms of AE (17), 32 patients with anti-NMDAR encephalitis were identified among 761 cases of encephalitis with uncertain etiology in the California Encephalitis Project between September 2007 and February 2011 (18), whereas 32 AE patients were identified among the 180 samples in our study. However, despite the usefulness of our findings, we still hope that a larger sample will be included in future studies. Furthermore, the scores and data for the model analysis for the APE² score were derived from the analysis of a single-center retrospective study. Therefore, the data need to be interpreted with caution, and a perspective prediction with the APE² score is needed in future studies.

Conclusions

The APE² score was validated as a predictive model with high sensitivity and specificity for AE antibodies to cell-surface proteins expressed in neurons. Based on the evidence presented here, we suggest that patients with an APE² score ≥5 undergo Ab testing. The use of an objective scoring system for suspected AE patients as early as possible can reduce the detection cost, improve the Ab detection rate, and allow timely and effective treatment.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by the Ethics Committee of the First Affiliated Hospital of Wenzhou Medical University. Written informed consent to participate in this study was provided by the participants' legal guardian/next of kin. Written informed consent was obtained from the individual(s), and minor(s)' legal guardian/next of kin, for the publication of any potentially identifiable images or data included in this article.

Author Contributions

SD conceived the study and drafted the article. JG collected the clinical data and interpreted the data. JL and YW performed the statistical analyses, transformed the raw data into figures and tables, and compiled the references. HX contributed to the writing of successive versions of the manuscript. All authors contributed with clinical assessment, data acquisition, analysis, diagnosis, and critical revision of the article.

Funding

This study is supported by the Wenzhou Science and Technology Plan Project (Grant No. Y20190096).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We thank all participating patients.

References

1. Joubert B, Dalmau J. The role of infections in autoimmune encephalitides. Rev Neurol. (2019) 175:420–6. doi: 10.1016/j.neurol.2019.07.004

CrossRef Full Text | Google Scholar

2. Graus F, Titulaer MJ, Balu R, Benseler S, Bien CG, Cellucci T, et al. A clinical approach to diagnosis of autoimmune encephalitis. Lancet Neurol. (2016) 15:391–404. doi: 10.1016/S1474-4422(15)00401-9

CrossRef Full Text | Google Scholar

3. Esposito S, Principi N, Calabresi P, Rigante D. An evolving redefinition of autoimmune encephalitis. Autoimmun Rev. (2019) 18:155–63. doi: 10.1016/j.autrev.2018.08.009

CrossRef Full Text | Google Scholar

4. Vincent A, Bien CG, Irani SR, Waters P. Autoantibodies associated with diseases of the CNS: new developments and future challenges. Lancet Neurol. (2011) 10:759–72. doi: 10.1016/S1474-4422(11)70096-5

CrossRef Full Text | Google Scholar

5. Frank L, Tha'is A, Josep D. Autoimmune encephalopathies. Ann N Y Acad Sci. (2015) 1338:94–114. doi: 10.1111/nyas.12553

CrossRef Full Text | Google Scholar

6. Zuliani L, Graus F, Giometto B, Bien C, Vincent A. Central nervous system neuronal surface antibody associated syndromes: review and guidelines for recognition. J Neurol Neurosurg Psychiatry. (2012) 83:638–45. doi: 10.1136/jnnp-2011-301237

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Dubey D, Alqallaf A, Hays R, Freeman M, Chen K, Ding K, et al. Neurological autoantibody prevalence in epilepsy of unknown etiology. JAMA Neurol. (2017) 74:397–402. doi: 10.1001/jamaneurol.2016.5429

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Dubey D, Kothapalli N, McKeon A, Flanagan EP, Lennon VA, Klein CJ, et al. Predictors of neural-specific autoantibodies and immunotherapy response in patients with cognitive dysfunction. J Neuroimmunol. (2018) 323:62–72. doi: 10.1016/j.jneuroim.2018.07.009

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Dubey D, Singh J, Britton JW, Pittock SJ, Flanagan EP, Lennon VA, et al. Predictive models in the diagnosis and treatment of autoimmune epilepsy. Epilepsia. (2017) 58:1181–9. doi: 10.1111/epi.13797

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Dubey D, Sawhney A, Greenberg B, Lowden A, Warnack W, Khemani P, et al. The spectrum of autoimmune encephalopathies. J Neuroimmunol. (2015) 287:93–7. doi: 10.1016/j.jneuroim.2015.08.014

CrossRef Full Text | Google Scholar

11. Graus F, Delattre JY, Antoine JC, Dalmau J, Giometto B, Grisold W, et al. Recommended diagnostic criteria for paraneoplastic neurological syndromes. J Neurol Neurosurg Psychiatry. (2004) 75:1135–40. doi: 10.1136/jnnp.2003.034447

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Lladó A, Mannucci P, Carpentier AF, Paris S, Blanco Y, Saiz A, et al. Value of Hu antibody determinations in the follow-up of paraneoplastic neurologic syndromes. Neurology. (2004) 63:1947–9. doi: 10.1212/01.WNL.0000144340.03364.BF

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Höftberger R, van Sonderen A, Leypoldt F, Houghton D, Geschwind M, Gelfand J, et al. Encephalitis and AMPA receptor antibodies: novel findings in a case series of 22 patients. Neurology. (2015) 84:2403–12. doi: 10.1212/WNL.0000000000001682

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Höftberger R, Titulaer MJ, Sabater L, Dome B, Rózsás A, Hegedus B, et al. Encephalitis and GABAB receptor antibodies: novel findings in a new case series of 20 patients. Neurology. (2013) 81:1500–6. doi: 10.1212/WNL.0b013e3182a9585f

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Gresa-Arribas N, Titulaer MJ, Torrents A, Aguilar E, McCracken L, Leypoldt F, et al. Antibody titres at diagnosis and during follow-up of anti-NMDA receptor encephalitis: a retrospective study. Lancet Neurol. (2014) 13:167–77. doi: 10.1016/S1474-4422(13)70282-5

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Petit-Pedrol M, Armangue T, Peng X, Bataller L, Cellucci T, Davis R, et al. Encephalitis with refractory seizures, status epilepticus, and antibodies to the GABAA receptor: a case series, characterisation of the antigen, and analysis of the effects of antibodies. Lancet Neurol. (2014) 13:276–86. doi: 10.1016/S1474-4422(13)70299-0

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Liu X, Yan B, Wang R, Li C, Chen C, Zhou D, et al. Seizure outcomes in patients with anti-NMDAR encephalitis: a follow-up study. Epilepsia. (2017) 58:2104–11. doi: 10.1111/epi.13929

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Gable MS, Sheriff H, Dalmau J, Tilley DH, Glaser CA. The frequency of autoimmune N-methyl-D-aspartate receptor encephalitis surpasses that of individual viral etiologies in young individuals enrolled in the California encephalitis project. Clin Infect Dis. (2012) 54:899–904. doi: 10.1093/cid/cir1038

PubMed Abstract | CrossRef Full Text | Google Scholar