Hua Geng
Xuqin Chen- Neurology Department, Children's Hospital of Soochow University, Suzhou, China
Background and purpose: This study aimed to effectively identify children with drug-resistant epilepsy (DRE) in the early stage of epilepsy, and take personalized interventions, to improve patients' prognosis, reduce serious comorbidity, and save social resources. Herein, we developed and validated a nomogram prediction model for children with DRE.
Methods: The training set was patients with epilepsy who visited the Children's Hospital of Soochow University (Suzhou Industrial Park, Jiangsu Province, China) between January 2015 and December 2017. The independent risk factors for DRE were screened by univariate and multivariate logistic regression analyses using SPSS21 software. The nomogram was designed according to the regression coefficient. The nomogram was validated in the training and validation sets. Internal validation was conducted using bootstrapping analyses. We also externally validated this instrument in patients with epilepsy from the Children's Hospital of Soochow University (Gusu District, Jiangsu Province, China) and Yancheng Maternal and Child Health Hospital between January 2018 and December 2018. The nomogram's performance was assessed by concordance (C-index), calibration curves, as well as GiViTI calibration belts.
Results: Multivariate logistic regression analysis of 679 children with epilepsy from the Children's Hospital of Soochow University (Suzhou Industrial Park, Jiangsu Province, China) showed that onset age<1, status epilepticus (SE), focal seizure, > 20 pre-treatment seizures, clear etiology (caused by genetic, structural, metabolic, or infectious), development and epileptic encephalopathy (DEE), and neurological abnormalities were all independent risk factors for DRE. The AUC of 0.92 for the training set compared to that of 0.91 for the validation set suggested a good discrimination ability of the prediction model. The C-index was 0.92 and 0.91 in the training and validation sets. Additionally, both good calibration curves and GiViTI calibration belts (P-value: 0.849 and 0.291, respectively) demonstrated that the predicted risks had strong consistency with the observed outcomes, suggesting that the prediction model in both groups was perfectly calibrated.
Conclusion: A nomogram prediction model for DRE was developed, with good discrimination and calibration in the training set and the validation set. Furthermore, the model demonstrated great accuracy, consistency, and prediction ability. Therefore, the nomogram prediction model can aid in the timely identification of DRE in children.
Introduction
Epilepsy is a disorder of the brain with the characteristic of an enduring predisposition to generate epileptic seizures, which has been recognized as one of the most prevalent neurological and psychiatric conditions by the World Health Organization (WHO). Currently, there have been more than 70 million patients with epilepsy worldwide (1), and its prevalence usually peaks among those <1 year old or > 50 years old (2). Three generations of antiepileptic drugs have been developed. Although AED therapy has played an effective role in stopping seizures in a great number of patients with epilepsy, and seizure control remains unsuccessful for some of them. As reported in studies, 20–30% become resistant to treatment with AEDs (3–5). Frequent seizures would cause neuron loss, axon germination, glial hyperplasia, and neural network remodeling (6), which could be fatal due to either the direct outcomes of seizures (such as sudden death from epilepsy, long falls, etc.) or the indirect outcomes of seizures (such as inhalation pneumonia, suicide, etc.) (7). Seizures seriously affect the development of the nervous system, resulting in growth retardation, low cognitive function, and a series of neuropsychological problems. The mortality rates of drug-resistant epilepsy (DRE) are four to seven times higher than those of patients with drug-sensitive epilepsy (DSE) (8). Kwan et al. indicated that some patients with DRE are possible to be identified early, and thus can be targeted for rational therapies or surgeries (9). Notably, the early risk factors for DRE, when paired with the prediction model, may offer a basis for personalized treatment. However, the early risk factors for DRE are still debatable, and there is no effective tool for diagnosing DRE. Therefore, it is vital to develop an early prediction model of DRE that will not only offer a basis for early DRE diagnosis and treatment selection but will also increase doctor–patient communication and achieve optimal allocation of healthcare resources. Most of the existing prediction models have focused on identifying independent risk factors for DRE (10). Scholars have developed several tools, including formulas, machine learning methods, prediction models of logistic regression, clinical prediction rules (CPR), deep learning methods, molecular diagnosis model, and integrative prediction algorithm based on combined clinical-EEG functional connectivity features and circulating microRNAs from plasma small extracellular vesicles, for the early prediction of the probability of DRE (11–18). Nevertheless, all of these tools were neither simple enough nor convenient for practical application. Nomogram is an intuitive prediction model which is graphically presented. Each independent risk factor listed in the nomogram can be quantified. Benefiting from these advantages, nomogram has been widely used in the diagnosis and prognosis of diseases. Thus, our team decided to develop a nomogram prediction model for DRE. Every potential risk factor in the nomogram model would be analyzed and quantified, to individually predict the likelihood of DRE in children. Meanwhile, we would assess the discrimination and calibration of the nomogram prediction model in internal and external children with DRE, to provide a reliable tool that could be directly used in clinical practice.
Materials and methods
Patients
The predictive nomogram model was developed based on a retrospective study of children with epilepsy who visited the Children's Hospital of Soochow University (Suzhou Industrial Park, Jiangsu Province, China) between January 2015 and December 2017. Internal validation was conducted using bootstrapping analyses with 1,000 resamples. The external validation set comprised patients with epilepsy from the Children's Hospital of Soochow University (Gusu District, Jiangsu Province, China) and the Yancheng Maternal and Child Health Hospital between January 2018 and December 2018. All the patients were regularly followed up in the Department of Neurology for more than 36 months.
Inclusion criteria
Currently, according to the guideline of the International League against Epilepsy (ILAE) 2010 (19), DRE is defined as a failure of adequate trials of two tolerated, appropriately chosen, and used antiepileptic drug schedules (whether administered alone or in combination) to achieve sustained freedom from seizure for three times the longest interval between seizures before the intervention (identified from seizures happened during the past 12 months), or 12 months. Thus, the inclusion criteria of DSE were as follows: (1) two unprovoked epileptic seizures more than 24 h apart; (2) sustained freedom from seizure for three times the longest interval between seizures before the intervention (identified from seizures happened during the past 12 months), or 12 months.
All the patients in the training set and validation set must regularly be followed up in the Department of Neurology more than 36 months after diagnosis.
Exclusion criteria
Exclusion criteria for DRE were: (1) incomplete information: children with incomplete clinical and follow-up information; (2) diagnostic mistakes: children mistakenly diagnosed as epilepsy, such as vasovagal syncope, migraine, transient ischemic attack, cardiogenic attack, and arrhythmia; (3) treatment errors: children failed to receive AEDs corresponding to their types of epilepsy, with inappropriate pharmacokinetic effects; (4) dose errors: for AEDs whose blood concentration can be measured, their doses were too low, that is, blood concentration did not reach the therapeutic level, or the AED could not be tolerated by children; for AEDs whose blood concentration cannot be measured, their doses did not reach the minimum maintenance dose recommended by the International League Against Epilepsy (ILAE) (20) or the instructions of the manufactures; (Supplementary Table 2); (5) inducing factors of epilepsy: poor medication compliance; (6) other serious systemic diseases: such as severe liver and kidney diseases, or congenital heart disease.
Exclusion criteria for DSE were: (1) poor treatment compliance; (2) acute symptomatic attacks; (3) missing information: incomplete clinical and follow-up data; (4) severe systemic diseases.
Data collection
According to the process, we collected the electronic medical histories of children with epilepsy in the past. Missing information was retrieved by phone call or directly contacting the children's guardians. The data in the questionnaire were systematically sorted and summarized. The personnel who were responsible for the collection and data input of medical history were trained by “Galaxy Plan” (which was a capacity-building project for professional doctors in epilepsy, held by the Chinese Anti-epilepsy Association), and obtained qualification certification, to ensure the accuracy and authenticity of data collection. Clinical history was obtained only from the child's legal guardian or long-term caregiver. Moreover, the original medical records and auxiliary examinations, as well as electronic medical records, were inquired for information validation, to ensure the authenticity and integrity of the data and to minimize recall bias. The information on explanatory variables were collected and documented at the time of the initial presentation of the patients to the study hospital. All the patients in the training set were regularly followed up in the Department of Neurology for more than 48 months after diagnosis. All patients were followed up for at least 36 months to guarantee sufficient time for the observation of therapeutic effect, disease progression, and outcome. This study complied with the principles of the Declaration of Helsinki, as well as its amendments, and was approved by the ethical committees of Children's Hospital of Soochow University and Yancheng Maternal and Child Health Hospital.
Definition of variables
1) Gender: male or female.
2) Onset age: within 1 year of birth or more than 1 year old.
3) Status epilepticus (SE): with or without status epilepticus.
4) Focal seizure: The children with DRE in the research were categorized and grouped into three categories: focal seizure, generalized seizure, and unclassified seizure, based on the ILAE classification of epilepsy in 2017, as well as the symptoms of seizures and EEG results.
5) Clustered seizures: two or more seizures within 24 h.
6) Pre-treatment seizures: it was defined as the number of seizures before the treatment and was divided into > 20 and ≤ 20 times (clustered seizures in one day means one seizure).
7) Radiological abnormalities: According to MRI in Epilepsy by Horst Urbach (21), head MRI was divided into normal conditions and abnormal ones related to epilepsy. The head MRI abnormalities included: hippocampal sclerosis, epilepsy-related tumor, cortical malformations, neurocutaneous syndrome, traumatic change, vascular malformation, ischemic changes, infectious and inflammatory changes, and metabolic changes.
8) Etiology: According to ILAE's suggestion, the etiology of DRE was divided into six types, namely, genetic, structural, metabolic, immune, infectious, and unknown ones (22).
9) Development and epileptic encephalopathy (DEE) include three electroclinical entities: developmental encephalopathy (DE: in conditions where the cognitive development and behavior are impaired independently of the epilepsy onset), epileptic encephalopathy (EE: it is a progressive brain dysfunction caused by frequent epileptic seizures and/or epileptic discharge), as well as the combination of the two (DEE). In clinical practice, it was difficult to distinguish DE from patients with EE, thus patients with DE, EE, or both were included in this study and recognized as patients with DEE (22–24).
10) Initial EEG: Using a Nihon Kohden EEG-1200C (Tokyo, Japan) or Nicolet EEG-V36 (USA), the EEG electrodes were placed on the scalp, recording voltage potentials resulting from current flow in and around neurons, according to the International 10–20 system. The video EEG of the patient was monitored and recorded for no less than 4 h. According to Clinical Electroencephalography by Xiao-yan Liu (25), pediatric EEG could be divided into normal and abnormal ones. EEG abnormalities included inter-seizure abnormalities, abnormalities during seizures, and background abnormalities. The video EEG before the treatment was called the initial EEG.
11) Neurological conditions: Physical examinations of the neurological system would be conducted for all the children. The results would be divided into normal and abnormal ones. Neurological abnormalities included abnormalities in motor function, sensory function, nerve reflex, skin, and skull.
12) Perinatal asphyxia: with or without perinatal asphyxia. The definition of perinatal asphyxia in China provided by the neonatal committee of the Chinese Medical Doctor Association (20) was as follows: (1) Prenatal high-risk factors that may lead to asphyxia; (2) Apgar score ≤7 in 1 or 5 min, without spontaneous respiration established. (3) PH < 7.15 in the umbilical arterial blood sample; (4) other causes of low Apgar score were excluded.
13) History of febrile convulsion: with or without history of febrile convulsion. Patients who have ever had febrile convulsion for once were defined to have a history of febrile convulsion.
14) Family history: with or without a family history. Family history refers to epilepsy in a first-degree relative or sibling.
15) Multiple seizure forms: according to the ILAE classification (22), seizure forms ≥2 were defined as multiple seizure forms.
Statistical analysis
Statistical analysis was performed by SPSS 21.0. The age and onset age in this study were not normally distributed, and thus were described as the median (interquartile interval). The categorical data were described as frequency (percentage). The continuous variables between groups were compared by the Mann–Whitney U test, and categorical variables were compared by the Chi-square (χ2) test. The risk factors were analyzed by univariate and multivariate logistic regression. The risk factors with P < 0.05 in the univariate analysis were deemed to indicate statistical significance, selected into the multivariate logistic regression analysis, and further screened by a stepwise forward method. Variance inflation factor (VIF), tolerance, and eigenvalue conditional indexes were used to detect collinearity.
A nomogram prediction model of individual DRE was developed with the independent risk factors of the multivariate regression analysis, using the rms, foreign, and gglot2 packages in R software. The corresponding scores for each predicting factor were listed in the nomogram, and then the score for all variables was summed up, mapping to a scale of outcomes that corresponded to the likelihood of DRE.
We tested the nomogram's performance through discrimination along with calibration. The nomogram was validated in the training and validation sets. The nomogram for DRE was subjected to 1,000 bootstrap resamples for internal validation to assess their predictive accuracies. The discrimination of the model was assessed through the receiver operating characteristic (ROC) curve, which was drawn using R software. A two-tailed P<0.05 was considered to be statistically significant. The nomogram's predictive accuracy was assessed using concordance (C-index), calibration curves along with GiViTI calibration belts. C-index's value is related to the predictive accuracy rate. The larger the C-index is, the higher the predictive accuracy rate would be. In addition, calibration plots of predictive DRE risk were assessed in the training and external validation sets. A perfect prediction would correspond to the 45° dashed line. The closer to the 45° dashed line, the predicted calibration curve is the better the predictive ability of the nomogram would be. GiViTI has been proposed as a graphical tool to identify ranges of probability where a model based on dichotomous outcomes mis-calibrates and applied to determine the goodness-of-fit of the prediction model (26). It has a calibration curve and a confidence belt. The calibration curve interprets the link of predicted risk with the observed outcome for various levels of risk, whereas the curve's confidence belt estimates the degree of uncertainty based on the true location of the curve. In the GiViTI calibration belt, after fitting a polynomial logistic function to the logit transformation of the predicted probability and outcome, the relationship between the predicted and observed outcomes is calculated. Through the calibration belt, 80% CI (marked in light gray) along with 95% CI (marked in dark gray) around the calibration curve are calculated. There would be a deviation from the bisector vector (dashed line) with statistical significance if the 95% CI does not cross the bisector (27).
Results
Baseline characteristics
A total of 856 children with epilepsy, who visited the Children's Hospital of Soochow University (Suzhou Industrial Park, Jiangsu Province, China) from January 2015 to December 2017, were evaluated, among which 177 children were excluded according to the inclusion criteria and exclusion criteria. Of the remaining 679 cases that were included in the training set for model development (407, 170, and 102 in preschool, school-age, and adolescence period, respectively), 229 belonged to the DRE group, and 450 were in the DSE group. Furthermore, there were a total of 347 patients included for external validation, including 189 children with epilepsy (64 in the DRE group, and 125 in the DSE group) who visited the Children's Hospital of Soochow University (Gusu District, Jiangsu Province, China) and 158 children with epilepsy (54 in the DRE group, and 104 in the DSE group) who visited Yancheng Maternal and Child Health Hospital from January 2018 to December 2018 (Figure 1). All the patients in the external validation set were regularly followed up in the Department of Neurology for at least 36 months from diagnosis of epilepsy. Patients with DSE in the external set were effectively controlled and their antiepileptic drugs were gradually withdrawn without recurrence. Finally, 1,026 children in total were included in this study, among which 679 children were in the training set, with 369 male (54.3%) and 310 female (45.7%), while 347 children were in the external validation set, with 200 male (57.6%) and 147 female (42.4%). There was no significant difference between the baseline characteristics of the two sets (Table 1). All the patients in the training set were regularly followed up in the Department of Neurology for more than 48 months after diagnosis, with the median follow-time reaching up to 66 months (range 48–84 months, interquartile range [IQR] 55.2–76.8 months), which was longer than the median of 42 months in the validation set (range, 36–48 months, IQR 38.4–45.6 months).
Figure 1. Technology roadmap.
Table 1. Baseline characteristics of the training set and validation set.
Model establishment
Univariate analysis showed that onset age <1, status epilepticus (SE), focal seizure, clustered seizures, >20 pre-treatment seizures, radiological abnormalities, clear etiology (caused by genetic, structural, metabolic, or infectious), DEE, abnormalities of initial EEG, perinatal asphyxia, neurological abnormalities, and multiple seizure forms were risk factors for DRE with statistical significances. However, gender, family history, and history of febrile convulsions were not associated with DRE (Table 2). Based on the collinearity diagnosis of 12 risk factors, we found the variance inflation factor (VIF) was <2, the tolerance was >0.1, the eigenvalue was close to 0, and the conditional indexes were <30. Thus, no collinearity in these risk variables could be deduced (Table 3 and Supplementary Table 1). Children with radiological abnormalities, moreover, are frequently characterized by symptomatic epilepsy, and children with clustered seizures are more prone to have >20 pre-treatment seizures. Taking this into consideration, the risk factors radiological abnormalities and clustered seizures were excluded in this study. The risk factors for DRE with statistical significance in the univariate analysis were further analyzed with unconditional binary multifactor logistic regression, and the result showed that onset age <1, SE, focal seizure, >20 pre-treatment seizures, clear etiology (caused by genetic, structural, metabolic or infectious), DEE, and neurological abnormalities were independent risk factors for DRE (Table 2). Therefore, seven independent risk factors were adopted and analyzed in the nomogram for predicting the incidence of individual DRE (Figure 2). For example, if there was an epileptic child aged 6 months old, with SE, structural etiology, 10 pre-treatment seizures, but without focal seizure, DEE, or neurological abnormalities, then the cumulative score of predicting factors was 40 + 45 + 99 + 0 + 0 + 0 + 0 = 184, and the corresponding likelihood for DRE was 0.84 (84%), indicating that this child was at 84% possibility for DRE (Figure 3).
Table 2. Univariate and multivariate logistic regression in the training set.
Table 3. Variance inflation factor (VIF) and tolerance analysis of variables.
Figure 2. The nomogram prediction model of individual DRE. SE, status epilepticus; DEE, development and epileptic encephalopathy. The top row is used for point assignments for each variable. Rows 2–8 indicated the variables included in the nomogram. The sum of the points is located on the total points line. The lowest row showed the diagnostic possibility of DRE.
Figure 3. Example of the nomogram prediction model for individual DRE. If there was an epileptic child aged 6 months old, with SE, structural etiology, 10 pre-treatment seizures, but without focal seizure, DEE, or neurological abnormalities, then the cumulative score of predicting factors was 40 + 45 + 99 + 0 + 0 + 0 + 0 = 184, and the corresponding likelihood for DRE was 0.84 (84%), indicating that this child was at 84% possibility for DRE.
Model validation
We tested the nomogram's performance through discrimination and calibration. The ROC curve for DRE prediction was drawn by R software. The AUC value was 0.92 for the training set compared to 0.91 for the validation set. The AUC of the nomogram model for both sets was >0.75, suggesting good discrimination for the nomogram prediction model (Table 4, Figure 4).
Table 4. AUCs of ROC curves for the nomogram and variables from the logistic regression model in the training as well as validation sets.
Figure 4. ROC curves for training and validation sets in the nomogram prediction model. (A) Training set. (B) Validation set (AUC = 0.92, AUC = 0.91). The area under the ROC curve was 0.92 and 0.91 in the training set and the validation set, respectively.
Internal validation of the nomogram was conducted through bootstrap analyses with 1,000 resamples. The C-index was 0.92 (95% CI 0.92–0.93) and 0.91 (95% CI 0.90–0.91) in the training and validation sets, respectively. Based on the two calibration curves, a good agreement was demonstrated between the predicted and observed values for DRE (Figure 5). The 80% and 95% CI of the GiViTI calibration belt did not cross the diagonal line, presenting P-values of 0.849 and 0.291 for the training set and for the validation set, respectively (P > 0.05), with no statistical significance (Figure 6). We also calculated the accuracy rate for the nomogram in each validation set: the internal validation (accuracy: 0.87, 95% CI 0.88–0.90, sensitivity:0.93, specitivity:0.76) and the external validation (accuracy:0.87, 95%, CI 0.84–0.90, sensitivty:0.93, specificity: 0.76). These findings indicated that the validation set had strong consistency with the training set. Thus, the nomogram prediction model was perfectly calibrated.
Figure 5. The calibration curves for the nomogram. Calibration plots of predictive DRE risk in the training set (A) as well as the external validation set (B). The x-axis showed the nomogram-predicted probability, while the y-axis showed the actual probability of DRE. Perfect prediction would correspond to the 45° dashed line.
Figure 6. Calibration of the nomogram for DRE prediction in the training set along with the validation set. The 80% CI and 95% CI of the GiViTI calibration belt for either the training set (A) or validation set (B) did not surpass the diagonal line, with P-values of 0.849 for the training set compared to that of 0.291 for the validation set in the GiViTI calibration belt (P > 0.05).
Discussion
Epilepsy is a brain disease persistently prone to result in epileptic seizures (28). The incidence of epilepsy in children was 144/100 000 and 58/100 000 person-years for those aged <1 year and 1–10 years, respectively. The cumulative incidence of epilepsy reached up to 0.66% for the children at 10 years old, among which 0.62% with active epilepsy (29). DRE accounts for 20–30% of all patients with epilepsy (3–5). As reported, there are 13 million disability-adjusted life years due to epilepsy each year globally (30). Long-term recurrent seizures exert a serious influence on the physical/mental health of children, as well as their quality of life. If the children at high risk of DRE could be effectively identified at the early stage, personalized interventions could be adopted to prevent its occurrence, reduce severe comorbidities, and improve prognosis. The risk factors for DRE currently reported have been inconsistent, if not controversial. Furthermore, most prediction models lack high replicability or sufficient evidence, and risk variables were not quantified specifically. Based on the above, it is of great significance to explore the risk factors for DRE among epileptic children and establish an easy-to-operate visual scale for the assessment of therapeutic efficacy. In our study, the established nomogram prediction model was found to be capable of identifying patients with DRE at diagnosis before the treatments, and the results suggested that this prediction model was able to identify patients with high-risk DRE at least 1 year earlier than the current practice that waits for failing from two AEDs.
Risk factors for DRE
In this study, compared to the 450 children with DSE, the independent risk factors for 229 children with DRE included onset age <1, SE, focal seizure, >20 pre-treatment seizures, clear etiology (caused by genetic, structural, metabolic, or infectious), DEE, and neurological abnormalities. Specifically, the risk of onset age < 1 year old in children with DRE was 3.96 times of those with onset age >1, which was consistent with other published literature (3). Infancy is the most critical period in brain development. Brain maturity may be related to the high incidence of seizures in infancy (31–33). Besides, the frequency of focal epilepsy in children with DRE was 2.88 times that of children with DSE in this study. As common focal epilepsy, hippocampal sclerosis has been recognized as the most common pathology contributing to drug-resistant mesial temporal lobe epilepsy (34), while patients with focal frontal epilepsy (35, 36) are prone to be drug-resistant. Thus, it can be inferred that patients with focal origins are more inclined to develop DRE. Additionally, we found that more than 20 pre-treatment seizures were an independent risk factor for DRE, which was consistent with the results of a 30-year longitudinal cohort study (37).
Regarding the etiology of epilepsy, epilepsy may be classified into structural, infectious, genetic, metabolic, immune, or unknown subgroups according to the suggestions of ILAE (22). After a series of auxiliary examinations, we were able to discover a clear and definitive etiology for the disease in the majority of the children with DRE, and some patients may have two or more reasons. When compared to those with an unclear etiology, epilepsy with a definite etiology was harder to treat and was more inclined to progress to DRE. Autoimmune encephalitis can cause epilepsy; however, it is not the risk factor of DRE. The prognosis of autoimmune encephalitis is correlated with the location of antigen and antibody, which can be divided into anti-neuron cell surface antibody and anti-neuronal intracellular antibody, the former is more common, and has a low probability of progressing to DRE, but the latter easily progress to DRE. In a word, anti-neuron cell surface antibody, the most common type of autoimmune encephalitis, is less likely to develop into DRE (38). The prognosis was poorest in patients with structural etiology, followed by metabolic, infectious, and genetic etiology, respectively; this differed from the results of previous studies based on the classification of idiopathic, symptomatic, and cryptogenic epilepsy (39, 40).
There are four types of drug resistance in clinical practice: de novo continuous drug resistance, reversal of drug resistance, possibly in an intermittent pattern in which periods of remission are followed by periods of uncontrolled seizures, and progression to drug resistance of delayed onset with persistent loss of efficacy after the control initially (41). Therefore, the condition of DRE may be recognized in more than 50% of early-stage patients. Focal signs implicating or localizing cerebral pathology are identified through the neurological examination (42), where dysmorphic features can suggest genetic syndromes (7). For instance, increased tone on one side of the body could suggest lesions in the contralateral hemisphere, such as cortical dysplasia (42). Stigmata of neurocutaneous syndromes are constituted by the facial port-wine stain of Sturge–Weber syndrome; café au lait spots and iris hamartomas of neurofibromatosis; and facial angiofibroma, periungual fibromas, hypomelanotic macules, as well as shagreen patches of tuberous sclerosis complex (TSC) (43). In our study, it was found that the risk factor of neurological abnormalities was an independent predictor for DRE, with its frequency in children with DRE5.72 times that of children with DSE. Multiple studies have proved the strong correlation between MRI abnormalities and the incidence of DRE (44–46), which was consistent with the result of our study. In our study, it was found that the risk of abnormalities on head MRI in children with DRE was 30.30 times higher than that in children with DSE (P < 0.001, OR = 30.30, 95% CI 18.43–49.82). However, as an auxiliary investigation, abnormalities on MRI were found to be collinear with the clinical etiology of DRE and were thus excluded by multiple logistic regression. Etiology was the biggest risk factor for DRE, according to the odds ratios in this study, followed by neurological abnormalities, SE, onset age <1, DEE, >20 pre-treatment seizures, and focal origins, respectively.
Establishment of a prediction model
The establishment of prediction models is conducive to the application of scoring tools in clinical guidelines, benefits doctor–patient communication, as well as provides an optimal allocation of healthcare resources. The essence of prediction models is to explore the law between independent and dependent variables through statistical analysis and to quantitate the relationship between predictive factors and outcomes. So far, prediction models mainly included formulas, scoring tables, web calculators, mobile APPs, nomograms, and so on. Nomogram achieves data visualization based on a complex mathematical formula. It makes use of the variables which are independent risk factors to picture a prediction model which is capable of quantifying a specific event's probability in clinical practice. After being listed separately, each variable is assigned a point, which is a weighted number based on the regression coefficient. Subsequently, the score for all the variables is summed up, followed by the matching to a scale to predict the outcomes. The formula can also be alternatively programming into a calculator based on the computer or smartphone. Once the variables are entered, the possibility of a specific event would be calculated (47). In other words, nomogram has the potential of transforming complicated regression equations into the visual figure, making prediction models easy-to-operate and allowing clinicians to conveniently obtain the likelihood of diseases. In recent years, nomogram has been an influencing element in healthcare decision-making at a system level. A well-established nomogram designed to solve a practical and confusing problem, if correctly interpreted or used, will be beneficial for clinicians, as well as patients. In this study, nomograms were drawn using R software (rms package) on the basis of the regression coefficients of independent risk factors for children with DRE. Throughout the establishment and validation of the model, we stand by the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) statement (48, 49). Because the nomogram can generate a numerical probability for individual DRE by integrating various independent and predictive risk factors, it fits our need model that combines demographic, genetic, biological, and clinical data. The clinicians no longer need to memorize complicated formulas. Nomograms can be beneficial for clinicians and patients not only in planning personalized treatments but also in providing a reference for decision-making. As far as we know, our nomogram model is very innovative for DRE prediction, since no relative visual tool has been established before.
Discrimination assessment of the nomogram prediction model
ROC curves have been widely used in the fields of medicine and machine learning since they are the most convenient evaluation tool for models. AUC was adopted to assess each predictor and how the model discriminates the patients with DRE from DSE. The value of AUC is associated with the model's discrimination ability. Generally speaking, the AUC between 0.5 and 0.75 means a poor ability, while AUC > 0.75 suggests a good ability (50). In this study, the AUC was 0.92 for the training set compared to 0.91 for the validation set, both larger than 0.75, suggesting a good discrimination ability for the nomogram prediction model.
Calibration assessment of the nomogram prediction model
The nomogram's performance was assessed by concordance (C-index), calibration curves along with GiViTI calibration belts. C-index exceeded 0.90 in all sets indicating good consistency of the nomogram. A prediction model is considered to be well-calibrated when it predicts the clinical event accurately. P-value of the GiViTI calibration belt < 0.05 indicates a not-so-perfect model. In this study, the 80% CI and 95% CI of the GiViTI calibration belt did not cross the dashed line (diagonal line), with P-values of 0.849 for the training set compared to that of 0.291 for the validation set in the GiViTI calibration belt (P > 0.05), indicating that the predicted risks had strong consistency with the observed outcomes, suggesting the perfect calibration of the prediction model.
Limitations of this study
This study also has shortcomings. To begin with, as this is a retrospective study, there may be a recall bias and, obviously, a selection bias. We used a strict inclusion criterion and collected sufficient samples to reduce bias. According to the results of PASS, the calculated sample size for the DSE and DRE groups was 186 children in total. However, the children included in this study were far more than the calculated minimum sample size. Second, the population for the model establishment came from a single center. Although we have applied external validations, the external validation was based on two hospitals; thus, data from other centers for more external validations are still necessary. Therefore, we are expecting other institutions to join this research and strengthen the prediction model in future.
Conclusion
1) The clinical information of children with DRE and DSE was retrospectively analyzed. The nomogram prediction model of individual DRE was constructed through univariate and multivariate logistic analyses.
2) Onset age < 1 year old, SE, focal seizure, >20 pre-treatment seizures, clear etiology (caused by genetic, structural, metabolic, or infectious), DEE, and neurological abnormalities were all independent risk factors for DRE.
3) Nomogram realized the visualization of the prediction model for DRE. Moreover, the model was well discriminated and calibrated among children, for model development and model validation. Based on the nomogram for DRE, we would have the potential not only to accurately predict the risk of individual DRE but also in achieving an early identification of DRE children. The predicted probability was strongly consistent with the real situation.
Data availability statement
The original contributions presented in the study are included in the article/Supplementary material, further inquiries can be directed to the corresponding author/s.
Ethics statement
Written informed consent was obtained from the individual(s), and minor(s)' legal guardian/next of kin, for the publication of any potentially identifiable images or data included in this article.
Author contributions
HG drafted the manuscript and contributed to the data acquisition. XC contributed to the conception and design of the study and critically revised and gave final approval for the publication of the article. Both authors contributed to the article and approved the submitted version.
Funding
This work was granted by the National Natural Science Foundation of China (82171441), Natural Science Foundation of Jiangsu Province (BK20201175), and Yancheng Science Development Project (YK2021006).
Acknowledgments
We would like to sincerely thank all the patients who kindly participated in this study.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fped.2022.905177/full#supplementary-material
References
1. Fisher RS, Acevedo C, Arzimanoglou A, Bogacz A, Cross JH, Elger CE, et al. ILAE official report: a practical clinical definition of epilepsy. Epilepsia. (2014) 55:475–82. doi: 10.1111/epi.12550
2. Thijs RD, Surges R, O'Brien TJ, Sander JW. Epilepsy in adults. Lancet. (2019) 393:689–701. doi: 10.1016/S0140-6736(18)32596-0
3. Kalilani L, Sun X, Pelgrims B, Noack-Rink M, Villanueva V. The epidemiology of drug-resistant epilepsy: A systematic review and meta-analysis. Epilepsia. (2018) 59:2179–93. doi: 10.1111/epi.14596
4. Mula M, Zaccara G, Galimberti CA, Ferro B, Canevini MP, Mascia A, et al. Validated outcome of treatment changes according to International League Against Epilepsy criteria in adults with drug-resistant focal epilepsy. Epilepsia. (2019) 60:1114–23. doi: 10.1111/epi.14685
5. Janmohamed M, Brodie MJ, Kwan P. Pharmacoresistance - Epidemiology, mechanisms, and impact on epilepsy treatment. Neuropharmacology. (2020) 168:107790. doi: 10.1016/j.neuropharm.2019.107790
6. Xiao Z, Peng J, Yang L, Kong H, Yin F. Interleukin-1beta plays a role in the pathogenesis of mesial temporal lobe epilepsy through the PI3K/Akt/mTOR signaling pathway in hippocampal neurons. J Neuroimmunol. (2015) 282:110–7. doi: 10.1016/j.jneuroim.2015.04.003
7. Devinsky O, Vezzani A, O'Brien TJ, Jette N, Scheffer IE, de Curtis M, et al. Epilepsy. Nat Rev Dis Primers. (2018) 4:18024. doi: 10.1038/nrdp.2018.24
8. Sperling MR. The consequences of uncontrolled epilepsy. CNS Spectr. (2004) 9:98–101. doi: 10.1017/S1092852900008464
9. Kwan P, Brodie MJ. Early identification of refractory epilepsy. N Engl J Med. (2000) 342:314–9. doi: 10.1056/NEJM200002033420503
10. Xue-Ping W, Hai-Jiao W, Li-Na Z, Xu D, Ling L. Risk factors for drug-resistant epilepsy: a systematic review and meta-analysis. Medicine. (2019) 98:e16402. doi: 10.1097/MD.0000000000016402
11. Boonluksiri P, Visuthibhan A, Katanyuwong K. Clinical prediction rule of drug resistant epilepsy in children. J Epilepsy Res. (2015) 5:84–8. doi: 10.14581/jer.15014
12. Delen D, Davazdahemami B, Eryarsoy E, Tomak L, Valluru A. Using predictive analytics to identify drug-resistant epilepsy patients. Health Inform. J. (2020) 26:449–60. doi: 10.1177/1460458219833120
13. Russo A, Posar A, Conti S, Parmeggiani A. Prognostic factors of drug-resistant epilepsy in childhood: an Italian study. Pediatr Int. (2015) 57:1143–8. doi: 10.1111/ped.12705
14. An S, Malhotra K, Dilley C, Han-Burgess E, Valdez JN, Robertson J, et al. Predicting drug-resistant epilepsy—a machine learning approach based on administrative claims data. Epilepsy Behav. (2018) 89:118–25. doi: 10.1016/j.yebeh.2018.10.013
15. Wang Y, Wang Y, Chen Y, Hua Y, Xu L, Zhu M, et al. Circulating microRNAs from plasma small extracellular vesicles as potential diagnostic biomarkers in pediatric epilepsy and drug-resistant epilepsy. Front Mol Neurosci. (2022) 15:823802. doi: 10.3389/fnmol.2022.823802
16. Wang B, Han X, Yang S, Zhao P, Li M, Zhao Z, et al. An integrative prediction algorithm of drug-refractory epilepsy based on combined clinical-EEG functional connectivity features. J Neurol. (2022) 269:1501–14. doi: 10.1007/s00415-021-10718-z
17. Smolyansky ED, Hakeem H, Ge Z, Chen Z, Kwan P. Machine learning models for decision support in epilepsy management: a critical review. Epilepsy Behav. (2021) 123:108273. doi: 10.1016/j.yebeh.2021.108273
18. Han T, Wu Z, Zhu J, Kou Y, Li J, Deng Y. Analysis and construction of a molecular diagnosis model of drug-resistant epilepsy based on bioinformatics. Front Mol Biosci. (2021) 8:683032. doi: 10.3389/fmolb.2021.683032
19. Kwan P, Arzimanoglou A, Berg AT, Brodie MJ, Allen Hauser W, Mathern G, et al. Definition of drug resistant epilepsy: consensus proposal by the ad hoc task force of the ILAE commission on therapeutic strategies. Epilepsia. (2010) 51:1069–77. doi: 10.1111/j.1528-1167.2009.02397.x
20. Patsalos PN, Berry DJ, Bourgeois BF, Cloyd JC, Glauser TA, Johannessen SI, et al. Antiepileptic drugs–best practice guidelines for therapeutic drug monitoring: a position paper by the subcommission on therapeutic drug monitoring, ILAE commission on therapeutic strategies. Epilepsia. (2008) 49:1239–76. doi: 10.1111/j.1528-1167.2008.01561.x
21. Horst Urbach. MRI in Epilepsy. New York: Springer (2013). ISBN: 978-3-642-25137-5. doi: 10.1007/174_2012_556
22. Scheffer IE, Berkovic S, Capovilla G, Connolly MB, French J, Guilhoto L, et al. ILAE classification of the epilepsies: position paper of the ILAE commission for classification and terminology. Epilepsia. (2017) 58:512–21. doi: 10.1111/epi.13709
23. Trivisano M, Specchio N. What are the epileptic encephalopathies? Curr Opin Neurol. (2020) 33:179–84. doi: 10.1097/WCO.0000000000000793
24. Scheffer IE, French J, Hirsch E, Jain S, Mathern GW, Moshe SL, et al. Classification of the epilepsies: new concepts for discussion and debate-special report of the ILAE classification task force of the commission for classification and terminology. Epilepsia Open. (2016) 1:37–44. doi: 10.1002/epi4.5
25. Xiao-yan Liu. Clinical Electroencephalography. Beijing: People's Medical Publishing House (2017). ISBN: 9787117244978.
26. Nattino G, Finazzi S, Bertolini G A. A new test and graphical tool to assess the goodness of fit of logistic regression models. Stat Med. (2016) 35:709–20. doi: 10.1002/sim.6744
27. Carvalho GMC, Leite TT, Liborio AB. Prediction of 60-day case fatality in critically Ill Patients receiving renal replacement therapy: external validation of a prediction model. Shock. (2018) 50:156–61. doi: 10.1097/SHK.0000000000001054
28. Fisher RS, van Emde Boas W, Blume W, Elger C, Genton P, Lee P, et al. Epileptic seizures and epilepsy: definitions proposed by the International League Against Epilepsy (ILAE) and the International Bureau for Epilepsy (IBE). Epilepsia. (2005) 46:470–2. doi: 10.1111/j.0013-9580.2005.66104.x
29. Aaberg KM, Gunnes N, Bakken IJ, Lund Søraas C, Berntsen A, Magnus P, et al. Incidence and prevalence of childhood epilepsy: A nationwide cohort study. Pediatrics. (2017) 139:e20163908. doi: 10.1542/peds.2016-3908
30. Collaborators GBDE. Global, regional, and national burden of epilepsy, 1990-2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet Neurol. (2019) 18:357–75. doi: 10.1016/S1474-4422(18)30454-X
31. Holmes GL, Ben-Ari Y. The neurobiology and consequences of epilepsy in the developing brain. Pediatr Res. (2001) 49:320–5. doi: 10.1203/00006450-200103000-00004
32. Holmes GL, Milh MD, Dulac O. Maturation of the human brain and epilepsy. Handb Clin Neurol. (2012) 107:135–43. doi: 10.1016/B978-0-444-52898-8.00007-0
33. Porter BE. Neurogenesis and epilepsy in the developing brain. Epilepsia. (2008) 49:50–4. doi: 10.1111/j.1528-1167.2008.01637.x
34. Malmgren K, Thom M. Hippocampal sclerosis–origins and imaging. Epilepsia. (2012) 53:19–33. doi: 10.1111/j.1528-1167.2012.03610.x
35. Nobili L, Francione S, Mai R, Cardinale F, Castana L, Tassi L, et al. Surgical treatment of drug-resistant nocturnal frontal lobe epilepsy. Brain. (2007) 130:561–73. doi: 10.1093/brain/awl322
36. Provini F, Plazzi G, Tinuper P, Vandi S, Lugaresi E, Montagna P. Nocturnal frontal lobe epilepsy. A clinical and polygraphic overview of 100 consecutive cases. Brain. (1999) 122:1017–31. doi: 10.1093/brain/122.6.1017
37. Chen Z, Brodie MJ, Liew D, Kwan P. Treatment outcomes in patients with newly diagnosed epilepsy treated with established and new antiepileptic drugs: a 30-year longitudinal cohort study. JAMA Neurol. (2018) 75:279–86. doi: 10.1001/jamaneurol.2017.3949
38. Wright S, Geerts AT, Jol-van der Zijde CM, Jacobson L, Lang B, Waters P, et al. Neuronal antibodies in pediatric epilepsy: clinical features and long-term outcomes of a historical cohort not treated with immunotherapy. Epilepsia. (2016) 57:823–31. doi: 10.1111/epi.13356
39. Proposal for revised clinical and electroencephalographic classification of epileptic seizures. From the commission on classification and terminology of the international league against epilepsy. Epilepsia. (1981) 22:489–501. doi: 10.1111/j.1528-1157.1981.tb06159.x
40. Proposal for revised classification of epilepsies and epileptic syndromes. Commission on classification and terminology of the international league against epilepsy. Epilepsia. (1989) 30:389–99. doi: 10.1111/j.1528-1157.1989.tb05316.x
41. Schmidt D, Loscher W. Drug resistance in epilepsy: putative neurobiologic and clinical mechanisms. Epilepsia. (2005) 46:858–77. doi: 10.1111/j.1528-1167.2005.54904.x
42. Stafstrom CE, Carmant L. Seizures and epilepsy: an overview for neuroscientists. Cold Spring Harb Perspect Med. (2015) 5:a022426. doi: 10.1101/cshperspect.a022426
43. Klar N, Cohen B, Lin DDM. Neurocutaneous syndromes. Handb Clin Neurol. (2016) 135:565–89. doi: 10.1016/B978-0-444-53485-9.00027-1
44. Covanis A. Epileptic encephalopathies (including severe epilepsy syndromes). Epilepsia. (2012) 53:114–26. doi: 10.1111/j.1528-1167.2012.03621.x
45. Seker Yilmaz B, Okuyaz C, Komur M. Predictors of intractable childhood epilepsy. Pediatr Neurol. (2013) 48:52–5. doi: 10.1016/j.pediatrneurol.2012.09.008
46. Ramos-Lizana J, Aguilera-Lopez P, Aguirre-Rodriguez J, Cassinello-Garcia E. Early prediction of refractory epilepsy in childhood. Seizure. (2009) 18:412–6. doi: 10.1016/j.seizure.2009.02.006
47. Balachandran VP, Gonen M, Smith JJ, DeMatteo RP. Nomograms in oncology: more than meets the eye. Lancet Oncol. (2015) 16:e173–80. doi: 10.1016/S1470-2045(14)71116-7
48. Royston P, Moons KG, Altman DG, Vergouwe Y. Prognosis and prognostic research: developing a prognostic model. BMJ. (2009) 338:b604. doi: 10.1136/bmj.b604
49. Altman DG, Vergouwe Y, Royston P, Moons KG. Prognosis and prognostic research: validating a prognostic model. BMJ. (2009) 338:b605. doi: 10.1136/bmj.b605
Keywords: drug-resistant epilepsy, prediction model, nomogram, drug-resistant epilepsy (DRE), children, drug-sensitive epilepsy (DSE)
Citation: Geng H and Chen X (2022) Development and validation of a nomogram for the early prediction of drug resistance in children with epilepsy. Front. Pediatr. 10:905177. doi: 10.3389/fped.2022.905177
Received: 26 March 2022; Accepted: 28 July 2022;
Published: 30 August 2022.
Edited by:
Rashmi Kumar, King George's Medical University, IndiaCopyright © 2022 Geng and Chen. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Xuqin Chen, drxuqinchen@163.com