External Validation of the WORSEN Score for Prediction the Deterioration of Acute Ischemic Stroke in a Chinese Population

Background: Early neurological deterioration (END) has been recognized as a serious neurological complication after acute ischemic stroke. However, to date, the WORSEN score was the only one scoring system specifically developed to detect END events in acute ischemic stroke patients. The purpose of this study was to investigate the WORSEN score's utility in China, and to determine the potential predictors of END in acute stroke patients.

Methods: Consecutive patients with acute ischemic stroke admitted to the Department of Neurology, Aerospace Center Hospital between March 2015 to February 2017 were recruited into the study's cohort and divided into two groups: patients with and without END. END was defined as either an increase in two or more NIHSS points, an increment of at least one point in motor power or a description of fluctuating of clinical symptoms in medical reports during the first 7 days after admission. Severe END was defined as an increase of NIHSS ≥ 4 points from baseline during the first 7 days after admission.

Results: Three hundred fifty four patients with acute ischemic stroke were enrolled in the present study and 67.5% were male. END occurred in 90 of these patients and severe END occurred in 55 of these patients. Logistic regression analysis showed that an initial NIHSS score ≥8, diameter of infarction, striatocapsular infarction, and TOAST type of large arterial atherosclerosis were independent predictors for END. The area under the ROC curve (AUC) of the WORSEN score for the prediction of END was 0.80 (95%CI 0.75–0.84), with a sensitivity of 62.22%, a specificity of 88.26%, positive predictive values of 64.37% and negative predictive values of 87.27%. Meanwhile, the AUC of the WORSEN score for the prediction of severe END was 0.82 (95%CI 0.78–0.86), with a sensitivity of 70.91%, specificity of 83.95%, positive predictive values of 44.83% and negative predictive values of 94.01%.

Conclusion: END is a relatively common neurological complication in patients with acute ischemic stroke. Our findings showed that the WORSEN score had a good predictive value for identifying patients with END in a Chinese population. Moving forward, multi-center studies are required for further validations

Introduction

Stroke is the second leading cause of death worldwide and is the cause of the highest disability-adjusted life-years lost in China (1). Early neurological deterioration (END), a relatively common neurological complication of stroke, occurs in 5–40% of acute ischemic stroke patients (2). Furthermore, it has been found that END is associated with poor outcomes (3, 4). Therefore, early detection of patients who are at risk of developing END is an important issue in clinical practice.

Many clinical and neuroimaging predictors have been associated with END. Simonsen et al. found that high blood glucose levels, the presence of large vessel disease, and a large perfusion lesion were associated with END (5). Yi et al. showed that patients with acute minor ischemic stroke with concomitant aspirin and clopidogrel resistance are at an increased risk for END (6) and Seners et al. reported that susceptibility vessel sign extension was an independent predictor of unexplained END in thrombolysed stroke patients (7).

However, when it comes to using scoring systems for assessing END in stroke patients, the WORSEN score is the only one that has been developed (Table 1). The WORSEN score (8), while not widely used, is based on definite clinical and imaging characteristics of stroke patients and is found to be a potentially valuable tool for detecting END in acute ischemic stroke patients. The purpose of this study was to investigate WORSEN scores' utility in our stroke population, and to determine the underlying predictor of END in acute stroke patients.

TABLE 1

Table 1. WORSEN score.

Materials and Methods

Subjects

This was a retrospective, observational study. Consecutive acute ischemic stroke patients admitted to the Department of Neurology, Aerospace Center Hospital between March 2015 and February 2017 were assessed. Patients were eligible for inclusion if they were 18 years or older, had a diagnosis of acute ischemic stroke defined in accordance with the World Health Organization criteria (9), and had symptom onset within 48 h. The diagnosis was verified by brain magnetic resonance imaging (MRI). Exclusion criteria of this present study were as follows: (1) age <18 years; (2) transient ischemic attack, intracerebral hemorrhage, subarachnoid hemorrhage, or brain tumors; (3) patients who accepted intravenous thrombolysis and(or) mechanical thrombectomy; (4) early discharge; (5) lack of data. This study was approved by the Ethics Committee of our hospital in accordance with the principles stated in the Declaration of Helsinki. Patient informed consent for inclusion in this study was waived.

Clinical Variables

For all patients, we collected the following clinical information: age; sex; disease history, such as hypertension, diabetes mellitus, dyslipidemia, smoking history, atrial fibrillation, coronary artery disease, stroke, and transient ischemic attack; NIH stroke scale (NIHSS) scores (10) at baseline; initial blood pressure; stroke subtype, according to Trial of Org10172 in Acute Stroke Treatment (TOAST) criteria (11). Laboratory blood tests, including full blood counts, fasting blood glucose, hemoglobin A1c (HbA1c), triglycerides, high-density lipoprotein cholesterol, low-density lipoprotein cholesterol, uric acid and homocysteine were also collected. The score of WORSEN was calculated according to Miyamoto et al. (8).

END and Severe END Definition

In order to draw attention to and encourage action to be taken in clinical practices, END was defined as an increase in two or more NIHSS points, an increment of at least one point in motor power, or description of fluctuating of clinical symptoms in medical reports during the first 7 days after admission (12–14). Severe END (15) was defined as an increase of NIHSS ≥ 4 points from baseline during the first 7 days after admission. Patients were classified into 2 groups according to the presence or absence of END.

Statistical Analysis

Continuous variables were presented as the means (standard deviation, SD) or medians (interquartile range, IQR). Comparisons between the two groups were performed using Student's t test or the Mann-Whitney U test after testing for normality. The χ² test or Fisher's exact test was conducted to analyze categorical variables. Logistic regression analysis was performed to evaluate variables' influence on END, adjusting for baseline variables when a p < 0.1 was found in the univariate analysis. We calculated the sensitivity and specificity of different levels of WORSEN scores for prediction of END (severe END) by using receiver operating characteristic (ROC) curves. All testing was two-tailed, and p < 0.05 were considered to be statistically significant. All data were analyzed using SPSS (Version 22.0, Windows, SPSS Inc., Chicago, IL).

Results

During the study period, 1,021 consecutive patients with acute ischemic stroke were potentially eligible for our study. After excluding patients that fit the exclusion criteria, 354 consecutive patients were finally included in the present study (Figure 1). The mean age was 64.2 ± 12.8 and 239 patients (67.5%) were male. END occurred in 90 of these patients (25.4%) and severe END occurred in 55 (15.5%). We identified the reasons for severe END in the 55 patients: four patients were due to hemorrhage transformation, seven patients due to malignant edema, ten patients due to recurrent stroke, seven patients due to medical complications, and 27 patients without a clear mechanism. We reported the demographics, baseline clinical characteristics and outcomes in patients with and without END (Table 2). There were statistically significant differences between the two groups (P < 0.05) in coronary disease, stroke etiology, percentage with an initial NIHSS score≥8, white blood count (WBC), neutrophil percentage, uric acid levels, striatocapsular infarction, diameter of infarct, and WORSEN score. However, there were no significant differences in other characteristics between the two groups. As expected, patients with END had worse modified Rankin Scale(mRS) scores at discharge than patients without.

FIGURE 1

Figure 1. Flow chat of the patients included in the present study.

TABLE 2

Table 2. The baseline demographics, clinical characteristics, and outcome between the patients with and without END.

Logistic regression analysis revealed that an initial NIHSS score ≥8 (odds ratio 2.64; 95%CI 1.13–6.19; P = 0.026), diameter of infarction (odds ratio 1.03; 95%CI 1.01–1.05; P = 0.011), striatocapsular infarction (odds ratio 2.18; 95%CI 1.13–4.19; P = 0.020), and TOAST type of large arterial atherosclerosis (odds ratio 2.87; 95%CI 1.13–7.30; P = 0.027) were independently associated with END (Table 3).

TABLE 3

Table 3. Logistic regression analysis for determinants of early neurological deterioration.

The area under the ROC curve (AUC) of the WORSEN score for the prediction of END was 0.80 (95%CI 0.75–0.84). Using Youden's index, the optimal cut point for a WORSEN score was 2, with a sensitivity of 62.22%, specificity of 88.26%, positive predictive values (PPV) of 64.37% and negative predictive values (NPV) of 87.27% (Figure 2). Meanwhile, the AUC of the WORSEN score for the prediction of severe END was 0.82 (95%CI 0.78–0.86); the optimal cut point for WORSEN score remained at 2 points, with a sensitivity of 70.91%, specificity of 83.95%, positive predictive values of 44.83% and negative predictive values of 94.01% (Figure 3). Table 4 shows the sensitivity, specificity, positive predictive value, negative predictive value and Youden's index for each WORSEN score in predicting severe END.

FIGURE 2

Figure 2. The predictive value of WORSEN score for END using ROC.

FIGURE 3

Figure 3. The predictive value of WORSEN score for severe END using ROC.

TABLE 4

Table 4. Sensitivity (Se), specificity (Sp), positive predictive value (PPV), negative predictive value (NPV), and Youden index for each WORSEN score in predicting severe early neurological deterioration.

Discussion

In our study, the incidence rates of END and severe END were 25.4 and 15.5% respectively. We found that an NIHSS score≥8, diameter of infarction, striatocapsular infarction, and TOAST type of large arterial atherosclerosis were independent predictors for END. WORSEN score showed the good predictive value in identifying patients with END after acute ischemic stroke in a Chinese population.

Although END has been well-recognized as a serious problem following acute ischemic stroke, the definition of END has been inconsistent among different studies and is still under constroversy (16). The reported incidence rate of total END in acute ischemic stroke patients without thrombolysis also has much uncertainty; ranging from 13.3 to 36.8% (2). Results from our study were in line with the two prior studies (6, 17), but were slightly higher than others (18, 19). This might be due to the fact that we adopted an increment in NIHSS scores≥2 points as the END definition. Siegler et al. had previously shown that the definition of END by an increase in NIHSS score of 2 or more points, as well as 4 or more points, were all associated with a poorer functional outcome at discharge (16). Our study showed that the mean mRS (3.34 ± 1.38) in the END group was significantly higher than in the non-END group (1.29 ± 0.80), which is consistent with the above findings. In the present study, we chose an increase in NIHSS score of 2 or more points as the definition of END because we wanted to advocate for early recognition and actions. However, some studies (2, 20) have pointed out that a small change in NIHSS score might be less meaningful and interrater dependent. Therefore, we also included an increase in NIHSS≥4 points for the severe END definition in our study.

With regard to the risk factors associated with END, hyperglycemia on admission (5), initial severity of stroke (19, 21, 22), mean platelet volume (17), metabolic syndrome (4), dehydration (23), presence of albuminuria (24) and fibrinogen (25), infarct size (19, 26) and proximal arterial occlusion (27) have been reported to be statistically significant in predicting END. Our findings that initial NIHSS score and infarct size are independently associated with END were consistent with previous studies, which may be explained by the fact that severe neurological impairment is a strong predictor of symptomatic intracranial hemorrhage (2) and malignant oedema (28). However, we did not find evidence that the blood sugar levels were associated with END, which is also in line with some other studies (17, 25, 28).

It is extremely important that clinicians are able to predicate the risk of END in patients with acute ischemic stroke in the early stages in order to choose an appropriate treatment method. Although many predictive models have been developed to project short-term or long-term functional outcomes of the disease and the risk of recurrence in patients (29–31), the WORSEN score (8) was the only one specifically developed for detection of END in acute ischemic stroke patients. Our study aimed to externally validate the WORSEN score in our population, and our findings indicated that it had a good predictive value for severe END with a sensitivity of 70.91%, specificity of 83.95%, and an the AUC of WORSEN score of 0.82 (95%CI 0.78–0.86).Fittingly, our findings were in line with prior studies.

Our study had several limitations. First, it was a single-center retrospective observational study. Patients with insufficient data for analysis were excluded and, despite the fact that included patients were consecutive, selection bias might have occurred and could have had a negative impact on our findings. Prospectively, large sample size and multi-center studies should be performed to confirm our findings.

Another limitation is in the various definitions of END. Despite the fact that we adopted the widely recommended definition of END and used logistic regression to build the predictive model, the possibility for residual confounding remains. Furthermore, although we excluded patients who accepted intravenous thrombolysis and (or) mechanical thrombectomy, we still could not eliminate the influence different treatments have on the outcomes of acute stroke patients. Finally, as the time of END has been recruited could be different for each patient, which might have also introduced bias.

In conclusion, END is a relatively common neurological complication in patients with acute ischemic stroke. Our study demonstrated that the WORSEN score had a good predictive power for identifying patients with END in a Chinese population. Moving forward, multi-center studies are required for further validate our findings.

Data Availability Statement

The datasets for this article are not publicly available as data collection is still in progress. Requests to access the datasets should be directed to YX, eWljaGVuZ3h1MjcmI3gwMDA0MDthbGl5dW4uY29t.

Ethics Statement

The studies involving human participants were reviewed and approved by the study was approved by the Institutional Ethical Committee of Aerospace Center Hospital. The ethics committee waived the requirement of written informed consent for participation.

Author Contributions

YX conceived the study, participated in the design, performed statistical analyses and drafted the manuscript. YC participated in the design, collected the data, and drafted the manuscript. RC performed statistical analyses and helped to draft the manuscript. FZ and PW participated in the statistical analyses. SY participated in the design and helped to revise the manuscript critically for important intellectual content. All authors read and approved the final manuscript.

Funding

This work was supported by the Ministry of Science and Technology of China (No. 2017YFC1307701).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We thank our colleague Siyu Yu, Wei Wang, Ge Zhang for their help of data collection.

References

1. Wu S, Wu B, Liu M, Chen Z, Wang W, Anderson CS, et al. Stroke in China: advances and challenges in epidemiology, prevention, and management. Lancet Neurol. (2019) 18:394–405. doi: 10.1016/S1474-4422(18)30500-3

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Seners P, Turc G, Oppenheim C, Baron JC. Incidence, causes and predictors of neurological deterioration occurring within 24 h following acute ischaemic stroke: a systematic review with pathophysiological implications. J Neurol Neurosurg Psychiatry. (2015) 86:87–94. doi: 10.1136/jnnp-2014-308327

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Helleberg BH, Ellekjaer H, Indredavik B. Outcomes after early neurological deterioration and transitory deterioration in acute ischemic stroke patients. Cereb Dis. (2016) 42:378–86. doi: 10.1159/000447130

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Zhang X, Sun Z, Ding C, Tang Y, Jiang X, Xie Y, et al. Metabolic syndrome augments the risk of early neurological deterioration in acute ischemic stroke patients independent of inflammatory mediators: a hospital-based prospective study. Oxid Med Cell Longev. (2016) 2016:8346301. doi: 10.1155/2016/8346301

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Simonsen CZ, Schmitz ML, Madsen MH, Mikkelsen IK, Chandra RV, Leslie-Mazwi T, et al. Early neurological deterioration after thrombolysis: clinical and imaging predictors. Int J Stroke. (2016) 11:776–82. doi: 10.1177/1747493016650454

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Yi X, Wang C, Liu P, Fu C, Lin J, Chen Y. Antiplatelet drug resistance is associated with early neurological deterioration in acute minor ischemic stroke in the Chinese population. J Neurol. (2016) 263:1612–9. doi: 10.1007/s00415-016-8181-5

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Seners P, Hurford R, Tisserand M, Turc G, Legrand L, Naggara O, et al. Is unexplained early neurological deterioration after intravenous thrombolysis associated with thrombus extension? Stroke. (2017) 48:348–52. doi: 10.1161/STROKEAHA.116.015414

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Miyamoto N, Tanaka R, Ueno Y, Watanabe M, Kurita N, Hira K, et al. Analysis of the usefulness of the WORSEN score for predicting the deterioration of acute ischemic stroke. J Stroke Cerebrovasc Dis. (2017) 26:2834–9. doi: 10.1016/j.jstrokecerebrovasdis.2017.07.005

PubMed Abstract | CrossRef Full Text | Google Scholar

9. The World Health Organization MONICA Project (monitoring trends and determinants in cardiovascular disease): a major international collaboration. WHO MONICA Project Principal Investigators. J Clin Epidemiol. (1988). 41:105–14. doi: 10.1016/0895-4356(88)90084-4

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Brott T, Adams HP Jr, Olinger CP, Marler JR, Barsan WG, Biller J, et al. Measurements of acute cerebral infarction: a clinical examination scale. Stroke. (1989) 20:864–70. doi: 10.1161/01.STR.20.7.864

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Adams HP Jr, Bendixen BH, Kappelle LJ, Biller J, Love BB, Gordon DL, et al. Classification of subtype of acute ischemic stroke. Definitions for use in a multicenter clinical trial. TOAST. Trial of Org 10172 in acute stroke treatment. Stroke. (1993) 24:35–41. doi: 10.1161/01.STR.24.1.35

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Kim JM, Bae JH, Park KY, Lee WJ, Byun JS, Ahn SW, et al. Incidence and mechanism of early neurological deterioration after endovascular thrombectomy. J Neurol. (2019) 266:609–15. doi: 10.1007/s00415-018-09173-0

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Kwon HM, Lee YS, Bae HJ, Kang DW. Homocysteine as a predictor of early neurological deterioration in acute ischemic stroke. Stroke. (2014) 45:871–3. doi: 10.1161/STROKEAHA.113.004099

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Berberich A, Schneider C, Reiff T, Gumbinger C, Ringleb PA. Dual antiplatelet therapy improves functional outcome in patients with progressive lacunar strokes. Stroke. (2019) 50:1007–9. doi: 10.1161/STROKEAHA.118.023789

CrossRef Full Text | Google Scholar

15. Zhou Y, Zhong W, Wang A, Huang W, Yan S, Zhang R, et al. Hypoperfusion in lenticulostriate arteries territory related to unexplained early neurological deterioration after intravenous thrombolysis. Int J Stroke. (2019) 14:306–9. doi: 10.1177/1747493019830595

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Siegler JE, Boehme AK, Kumar AD, Gillette MA, Albright KC, Martin-Schild S. What change in the national institutes of health stroke scale should define neurologic deterioration in acute ischemic stroke? J Stroke Cerebrovasc Dis. (2013) 22:675–82. doi: 10.1016/j.jstrokecerebrovasdis.2012.04.012

CrossRef Full Text | Google Scholar

17. Oji S, Tomohisa D, Hara W, Tajima T, Suzuki M, Saito A, et al. Mean platelet volume is associated with early neurological deterioration in patients with branch atheromatous disease: involvement of platelet activation. J Stroke Cerebrovasc Dis. (2018) 27:1624–31. doi: 10.1016/j.jstrokecerebrovasdis.2018.01.012

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Alexandrov AV, Felberg RA, Demchuk AM, Christou I, Burgin WS, Malkoff M, et al. Deterioration following spontaneous improvement : sonographic findings in patients with acutely resolving symptoms of cerebral ischemia. Stroke. (2000) 31:915–9. doi: 10.1161/01.STR.31.4.915

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Miyamoto N, Tanaka Y, Ueno Y, Kawamura M, Shimada Y, Tanaka R, et al. Demographic, clinical, and radiologic predictors of neurologic deterioration in patients with acute ischemic stroke. J Stroke Cerebrovasc Dis. (2013) 22:205–10. doi: 10.1016/j.jstrokecerebrovasdis.2011.07.018

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Josephson SA, Hills NK, Johnston SC. NIH stroke scale reliability in ratings from a large sample of clinicians. Cerebrovasc Dis. (2006) 22:389–95. doi: 10.1159/000094857

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Haeusler KG, Gerischer LM, Vatankhah B, Audebert HJ, Nolte CH. Impact of hospital admission during nonworking hours on patient outcomes after thrombolysis for stroke. Stroke. (2011) 42:2521–5. doi: 10.1161/STROKEAHA.110.612697

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Suda S, Katsumata T, Okubo S, Kanamaru T, Suzuki K, Watanabe Y, et al. Low serum n-3 polyunsaturated fatty acid/n-6 polyunsaturated fatty acid ratio predicts neurological deterioration in Japanese patients with acute ischemic stroke. Cerebrovasc Dis. (2013) 36:388–93. doi: 10.1159/000355683

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Lin LC, Yang JT, Weng HH, Hsiao CT, Lai SL, Fann WC. Predictors of early clinical deterioration after acute ischemic stroke. Am J Emerg Med. (2011) 29:577–81. doi: 10.1016/j.ajem.2009.12.019

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Kanamaru T, Suda S, Muraga K, Okubo S, Watanabe Y, Tsuruoka S, et al. Albuminuria predicts early neurological deterioration in patients with acute ischemic stroke. J Neurol Sci. (2017) 372:417–20. doi: 10.1016/j.jns.2016.11.007

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Lee SJ, Hong JM, Lee SE, Kang DR, Ovbiagele B, Demchuk AM, et al. Association of fibrinogen level with early neurological deterioration among acute ischemic stroke patients with diabetes. BMC Neurol. (2017) 17:101. doi: 10.1186/s12883-017-0865-7

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Davalos A, Toni D, Iweins F, Lesaffre E, Bastianello S, Castillo J. Neurological deterioration in acute ischemic stroke: potential predictors and associated factors in the European cooperative acute stroke study (ECASS) I. Stroke. (1999) 30:2631–6. doi: 10.1161/01.STR.30.12.2631

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Mori M, Naganuma M, Okada Y, Hasegawa Y, Shiokawa Y, Nakagawara J, et al. Early neurological deterioration within 24 hours after intravenous rt-PA therapy for stroke patients: the stroke acute management with urgent risk factor assessment and improvement rt-PA registry. Cerebrovasc Dis. (2012) 34:140–6. doi: 10.1159/000339759

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Strbian D, Meretoja A, Putaala J, Kaste M, Tatlisumak T Helsinki Stroke Thrombolysis Registry G. Cerebral edema in acute ischemic stroke patients treated with intravenous thrombolysis. Int J Stroke. (2013) 8:529–34. doi: 10.1111/j.1747-4949.2012.00781.x

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Pan Y, Jing J, Zhang R, Zhao X, Liu L, Wang H, et al. Poor performance of stroke prognostication using age and national institutes of health stroke scale-100 to predict 3- and 12-month outcomes of ischemic stroke in China national stroke registry. J Stroke Cerebrovasc Dis. (2014) 23:2335–40. doi: 10.1016/j.jstrokecerebrovasdis.2014.04.031

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Flint AC, Faigeles BS, Cullen SP, Kamel H, Rao VA, Gupta R, et al. THRIVE score predicts ischemic stroke outcomes and thrombolytic hemorrhage risk in VISTA. Stroke. (2013) 44:3365–9. doi: 10.1161/STROKEAHA.113.002794

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Ling X, Yan SM, Shen B, Yang X. A modified essen stroke risk score for predicting recurrent ischemic stroke at one year. Neurol Res. (2018) 40:204–10. doi: 10.1080/01616412.2018.1428389

PubMed Abstract | CrossRef Full Text | Google Scholar