Application Values of Six Scoring Systems in the Prognosis of Stroke Patients

Objective: The present study aimed to evaluate the prognostic value of Acute Physiology and Chronic Health Evaluation (APACHE; II and III), Chinese Stroke Scale (CSS), National Institutes of Health Stroke Score (NIHSS), activities of daily living (ADL) (Barthel index, BI), and Glasgow Coma Scale (GCS) scores for stroke patients.

Methods: A total of 352 stroke patients were evaluated using APACHE II, APACHE III, CSS, NIHSS, ADL, and GCS scores within 24 h after admission. And these patients were consecutive admissions to the hospital. The endpoint was in-hospital death. The scores of these scales were compared between the survival group and death group, and the receiver operating characteristic (ROC) curves were drawn. The ability of each scoring system to predict the prognosis of patients was evaluated using the area under the ROC curve, and the areas under the curves (AUCs) of these six scales were compared.

Results: The AUCs of the APACHE II, APACHE III, CSS, NIHSS, ADL, and GCS scores were 0.882, 0.867, 0.832, 0.859, 0.838, and 0.819, respectively.

Conclusion: APACHE II, APACHE III, CSS, NIHSS, ADL, and GCS scores have good predictive values in the prognosis of stroke patients. APACHE II is superior among the other five scales.

Introduction

Stroke is a common and prevalent disease which has the characteristics of high incidence, high mortality rate, high disability rate, high recurrence rate and many complications. Stroke, along with heart disease and malignant tumor, are the three leading causes of death in most countries, which bring heavy economic, and mental burden to families and the society. To date, there are many methods to assess stroke at home and abroad, but the most haven't been widely adopted. Moreover, in order to prove the value of strengthening medical treatment, evaluate the condition of patients with different diseases and compare the therapeutic effects of different treatment schemes, a unified assessment criterion is needed. Therefore, it is necessary to comprehensively evaluate some of the major stroke scoring methods presently used around the world. The investigators intended to compare Acute Physiology and Chronic Health Evaluation (APACHE) II and III (1, 2), Chinese Stroke Scale (CSS) (3), National Institutes of Health Stroke Score (NIHSS) (4), activities of daily living (ADL) (Barthel Index, BI) (5), and Glasgow Coma Scale (GCS) score (6). The same stroke patient was comprehensively assessed to compare the application values of these six scoring systems in the prognosis of stroke, providing reference for the correct selection of stroke scale.

Information and Methods

Patient Selection

A total of 352 patients with acute stroke, who were admitted in the Department of Neurology and Department of Neurosurgery, Affiliated Hospital of North China University of Science and Technology from January 2015 to December 2016 were enrolled in the present study. All selected patients were admitted to the hospital within 3 days after onset. The diagnoses of all patients were in accordance with the diagnostic criteria for cerebrovascular disease developed by the Fourth Academic Conference of National Cerebral Vascular Disease in 1995, and confirmed by head computed tomography (CT) or magnetic resonance imaging (MRI). The following patients were excluded from the study: patients with transient ischemic attacks; patients with mild conditions who had only sensory symptoms or a muscle strength not lower than grade IV; patients with serious dysfunction of heart, liver, kidney, and other organ which could become the main reason of influencing the prognosis of patients; patients who have a history of stroke, and could not take self-care of themselves; patients who did not comply or could not complete the tests; patients with hematological and neoplastic cerebral hemorrhage; patients with atherosclerotic cerebral infarction or cerebral embolism.

Evaluation Method

Stroke patients who met the inclusion criteria were evaluated using the following six scoring scales within 24 h after admission: APACHE II, APACHE III, CSS, NIHSS, ADL, and GCS scores. All receiver operating characteristic (ROC) curves of APACHE II, APACHE III, CSS, NIHSS, ADL, and GCS scores to predict the prognosis of patients were drawn, and the areas under the ROC curve (AUCs) were calculated. The area under the ROC curve of the two scoring systems was compared by the method come up with by Hanley and McNeil (7). The calculation formula and the concrete steps were as follows:

$\begin{array}{l}①\text{Z}\text{=}({\text{A}}_{\text{1}}-{\text{A}}_{\text{2}})\text{/}{({\text{SE}}_{\text{1}}^{\text{2}}{\text{+ SE}}_{\text{2}}^{\text{2}}-{\text{2rSE}}_{\text{1}}{\text{SE}}_{\text{2}})}^{\text{0}.\text{5}}\\ ②{\text{r}}^{\text{a}}\text{=}({\text{r}}_{\text{N}}{\text{+ r}}_{\text{A}})\text{/2}\\ ③{\text{A}}^{\text{a}}\text{=}({\text{A}}_{\text{1}}{\text{+ A}}_{\text{2}})\text{/2}\end{array}$

r_N was the related coefficient between the two scores of survival patients. r_A was the related coefficient between the two scores of the dead patients. r was the related coefficient of the area under the ROC curve of the two scores. A₁ was the area under ROC curve of score 1. A₂ was the area under ROC curve of score 2. SE₁ (Standard Error of Area 1) was the standard error of area under ROC curve of score 1. SE₂ (Standard Error of Area 2) was the standard error of area under ROC curve of score 2.

Using the values of r^a and A^a, the value of r was obtained by looking up tables. The above results were brought into the formula to obtain the Z-value of the normal distribution statistics. Z-score was obtained by looking up tables. When Z ≥ 1.96 and P < 0.05, it could be concluded that the area difference between the two scoring ROC curves was significant.

Statistical Analysis

All data were analyzed using SPSS 13.0 statistical software. Measurement data were compared using t-test. All tests were two-sided tests. P < 0.05 was considered statistically significant.

Results

Gender, Consciousness State, Lesion Sites, and Lesion Types in the Death Group and Survival Group (Table 1)

A total of 352 patients were enrolled into the present study. Among these patients, 196 patients were male and 156 patients were female. Furthermore, among these patients, 26 patients had lacunar infarction, 220 patients had cerebral infarction, 80 patients had cerebral hemorrhage, and 26 patients had mixed cerebrovascular disease. In addition, bilateral lesions were found in 112 patients, left lesions were found in 110 patients, right lesions were found in 106 patients, and brainstem lesions were found in 24 patients. Moreover, when they were admitted to hospital, 242 patients were conscious and awake, 38 patients were sleepy, 14 patients were lethargic, 22 patients were in mild coma, 16 patients were in moderate coma, and 20 patients were in deep coma.

TABLE 1

Table 1. Consciousness state composition of the survival group and death group [n (%)].

Comparison of Scores Between the Survival and Death Groups

Table 2 shows that on the first day of admission, the differences in APACHE II, APACHE III, CSS, NIHSS, GCS, and ADL scores of patients with acute stroke between the survival group and death group were statistically significant, the APACHE II, APACHE III, CSS, and NIHSS scores were significantly higher in the death group than in the survival group, and the GCS and ADL scores were significantly lower in the death group than in the survival group.

TABLE 2

Table 2. Comparison of scores in hospitalized patients with acute stroke between the survival group and death group.

Comparison of the Validity of Scoring Systems in Predicting the In-Hospital Mortality of Patients With Acute Stroke

Evaluating the validity of a scoring system in judging the prognosis of a disease can be confirmed through the ROC curve. First, with the actual in-hospital death or survival of patients with acute stroke as the gold standard for the prognosis of patients, the sensitivity and specificity of these scoring systems at each point were calculated. High APACHE II, APACHE III, CSS, and NIHSS scores reflect more severe stroke impairment while lower ADL and GCS scores reflect more severe stroke impairment. Therefore, ADL and GCS scores were used to construct the ROC curves, in order to conform to the principle that the higher the score, the more serious the disease became. The ROC curves of these scoring systems at admission were drawn (Figure 1). Baseline distributions of GCS, NIHSS were provided as scatter plots (Figure 2), the AUCs were calculated, and the differences between the AUC of these scoring systems and area under the baseline (0.5) were compared (Table 2). Comparisons of AUC were conducted between any pair of six scoring systems (Table 3). Based on the ROC curves, the cut-off values and the predictive power in predicting prognosis of stroke patients according to 6 scoring systems were listed in Table 4.

FIGURE 1

Figure 1. The ROC curve of six scores of ACVD patients on the 1 day of admission.

FIGURE 2

Figure 2. Scatter plot of baseline distributions of GCS, NIHSS.

TABLE 3

Table 3. Comparisons of AUC between two of six scoring systems of the prognosis of stroke patients.

TABLE 4

Table 4. Sensitivity and specificity of six scores in predicting prognosis of stroke patients.

Table 2 revealed that all these scoring systems had predictive value for the in-hospital mortality of patients with acute stroke, and had good validity. The ROC curves for the APACHE II, APACHE III, CSS, NIHSS, ADL, and GCS scores on the first day of admission were drawn, and the AUCs for the APACHE II, APACHE III, CSS, NIHSS, ADL, and GCS scores were 0.882, 0.867, 0.832, 0.859, 0.838, and 0.819, respectively. These results revealed that the APACHE II, APACHE III, CSS, NIHSS, ADL, and GCS scores were valuable in predicting the in-hospital mortality of patients with acute stroke. From Table 3, it could be concluded the AUC of APACHE II and GCS, CSS, and NIHSS, and between GCS and ADL had statistical significance, while AUC of other items in Table 3 had no statistical significance. Table 4 revealed that the score of Apache II was 10 points, Apache III was 26 points, CSS was 26 points, NIHSS was 14 points, 100-ADL was 80 points (ADL was 20 points) and 15-GCS was 4 points (GCS was 11 points) with the largest Jordan index. Therefore, the cutoff of APACHE II, APACHE III, CSS, NIHSS, ADL, and GCS scores were 10, 26, 26, 14, 20, and 11 points, respectively.

Discussion

Objective and accurate evaluation of the severity of the disease in critical patients is of great significance for the judgment of the disease. The purpose of the application of the scoring method for critical disease is to quantitatively evaluate the severity of disease, and predict the risk of disease or death of patients (8, 10, 11). This is based on patient indicators, such as acute physiological changes, anatomical changes, and even chronic disease lesion factors, and the severity of the disease can be objectively quantified by assigning values, weighting, logical reasoning, and complex mathematical operations (12). This quantitative evaluation result is very important for doctors to understand the condition and dynamically observe the changes in disease. For stroke patients, stroke scale is the only approach to solve this problem (13). Quantifying the prognosis of stroke patients can more accurately evaluate the severity of the disease, and predict the risk of death, providing a scientific basis for evaluating the condition of critical patients. According to these scores, one can judge the condition of the disease, and more reliably predict the mortality rate of a patient population, providing an objective basis for doctors, family members, and the society to make medical decisions, help correctly formulate the treatment plan and explain the condition to the family members. Furthermore, it can assess the selective operation of monitoring patients in the intensive care unit (ICU) (6), and is helpful in promoting the in-depth study of stroke. The evaluation of the neurological deficit and prognosis of stroke has been carried out for half a century, but there is presently no uniform standard.

The general pathogenesis of stroke is sudden vascular rupture or vascular obstruction in the brain, which causes interruption of the channels through which blood flows into the brain, induces ischemia and hypoxia in brain tissues, and subsequently damages brain tissues (14). A related study revealed that (15) stroke has become the first cause of death in China, and also the primary cause of disability in adults. A study revealed that the prognostic factors in stroke patients generally include health status, systemic reactions, and the severity and nature of intracranial lesions (16).

The APACHE scoring system is a commonly used scoring system, which includes health status, age and physiological scores. The APACHE III scoring system was expanded and developed based on the APACHE II scoring system, and these two have good consistency in death probability. The APACHE III scoring system fully takes the severity of visceral dysfunction, age and the patient's own functional status into account, which is commonly used to predict the severity of all types of patients (17, 18).

NIHSS is the most widely used scale for evaluating the severity of stroke in the world, and this has been mainly used to evaluate the severity of neurological deficit, curative effect and prognosis (19–22). At present, the NIHSS scoring system is commonly used to assess the severity of stroke patients and predict mortality in clinic, and helps physicians take active intervention measures to improve the prognosis of critical patients (23). The NIHSS scoring system only scores 11 indicators of neurological function, and neglects factors that affect the prognosis, such as age and chronic health status. Therefore, this has some limitations in judging the severity and prognosis of the disease (24).

The CSS score has been revised and modified based on the Scandinavian Stroke Scale (SSS) by Qingtang Chen et al., which was adopted at the Second National Conference on Cerebrovascular Diseases, and its projects were revised again at the Fourth National Conference on Cerebrovascular Diseases in 1995. Jing Su in China revealed that (25) the internal consistency of NIHSS in stroke patients is relatively high, which has predictive validity for the prognosis of stroke, and has the best structural validity in some patients with anterior circulation. Zirong Tao also considered that the CSS score has better reliability, validity and sensitivity in stroke patients (9). Furthermore, a study revealed that (26) the AUC of the CSS score for the prognosis of patients with acute cerebral infarction was 0.796.

The GCS score is a coma scale first developed for patients with cerebral surgery in 1974 by Teasdale et al. (27). This has also been commonly used in stroke patients, and was revised in 1976 (28). Clinicians usually use GCS to evaluate the level of consciousness state of stroke patients, and its assessment of nervous system functional defects focuses on uncommon symptoms of acute stroke patients, such as difficulty in eye closure, decerebration, and decortication symptoms, but these do not include aphasia and sports defect classification. For patients with stroke, GCS often overestimates the degree of neurological deficit. Therefore, it is not suitable for stroke patients without impaired consciousness, but for patients with aphasia.

BI was developed in the mid 1950s, and was designed and applied in clinic by Horence Mahoney and Dorothy Barthel in the United States. This was called the Maryland disability index at that time, was officially called the Barthel index (BI) in literatures in the mid 1960s, and is being used at present. It is simple in terms of evaluation, and has high credibility and sensitivity. Furthermore, it is one of the most widely used and most widely studied ADL evaluation methods, it can not only be used to assess the functional status before and after treatment, and can predict the outcomes of treatment, length of hospital stay and prognosis. Furthermore, it is a common method in rehabilitation medical institutions in the United States, and is also one of internationally recognized ADL evaluation methods (29). This index was first published by Dorothea Barthel and F Mahoney in 1965. It is an effective index to judge the recovery of limb function in stroke patients, and is a commonly used evaluation method in rehabilitation medicine. However, its prediction for stroke death has not been reported.

The APACHE II, APACHE III, CSS, NIHSS, ADL, and GCS scores are used to evaluate the condition of stroke patients. However, the comparison of predictive values among these six scoring methods in stroke patients has not been reported.

The present study revealed that the APACHE II, APACHE III, CSS, and NIHSS scores were significantly higher in the death group than in the survival group, and ADL and GCS scores were significantly lower in the death group than in the survival group. The differences in these six scoring systems between the survival group and death group were statistically significant (P < 0.001). These results reveal that all six scores can predict the prognosis of stroke patients, which is consistent with that reported in previous literatures. The present study also revealed that the AUCs for the APACHE II, APACHE III, CSS, NIHSS, ADL, and GCS scores were 0.882, 0.867, 0.832, 0.859, 0.838, and 0.819, respectively. It is general considered that the larger the area under the general recognition curve, the more authentic it is to predict the prognosis of patients. This suggests that all these six scores have good validity for the evaluation of the prognosis of stroke patients. Chen et al. considered that (30) the APACHE II score has important clinical significance in evaluating the prognosis of stroke patients. Bein et al. conducted a comparative study on the comparison between APACHE II and APACHE III in 150 patients in the ICU (31). The results revealed that both APACHE II and APACHE III have good prognostic values, and it was concluded that the area under the ROC curve for APACHE II was 0.847, and the area under the ROC curve for APACHE III was 0.899. A study in China confirmed that compared with the APACHE II scoring system, the APACHE III scoring system is more accurate in predicting death, and more reasonable and scientific in system design. Furthermore, the study also revealed that the APACHE III scoring system was more accurate in assessing the disease condition for patients with internal diseases (32). In addition, another study revealed that (33) for stroke patients, the AUC of APACHE III was 0.848, the sensitivity was 0.749, the specificity was 0.802, the standard error was 0.017, and the 95% confidence interval was 0.816–0.881. That study revealed that the application of the APACHE III scoring system could be applied for a more scientific and reasonable clinical evaluation of the condition and prognosis of acute stroke patients, providing a reliable basis for actively implementing intervention, effectively improving the prognosis of patients, and increasing the survival rate of patients. In addition, we evaluated the scores on the first day, the third day and the seventh day of admission. Through statistical analysis, we calculated the predictive value of various scores at different time points. The score and node with the largest predictive value were determined by comparing the area under ROC cur so as to predict the prognosis of the patients with acute stroke more accurately in the future. This is the rationale why 24 h was used as the reference timepoint.

Quantifying the prognosis of stroke patients can accurately evaluate the severity of the disease, predict the prognosis of the patients, help to make a correct treatment plan and explain the condition to the family members. It has been generally considered that the greater the AUC, the higher the validity for the prediction of the prognosis of patients. In clinical practice, the area under ROC curve of prediction index is usually between 1 and 0.5. By comparing the area under ROC curve, the clinician can intuitively know whether an index has reference value in disease prediction and can compare the prediction efficiency of each index by statistical method.

Therefore, the present study considered that the APACHE II and APACHE III scores in predicting the prognosis of patients with acute stroke should be emphasized.

However, our study still have some limitations. Firstly, the result of scoring systems may be influenced by subjective factors to a great extent. So our research results need to be verified by prospective research of large sample. Secondly, this study was still in the initial stage of research, and lacked comprehensive, systematic and large-scale clinical dynamic assessment, which was also the direction of our future research.

In summary, scale assessment has been widely applied in clinical and scientific research, which plays a great role in promoting the improvement of medical level. Each scale has its own application range, as well as its own advantages and disadvantages. No scale can completely measure and predict all types of strokes, and all changes in its indicators. Therefore, the accurate selection and application of the appropriate evaluation scale play an important role in judging the condition, making treatment decisions, and evaluating the treatment efficacy and prognosis of the disease. There is presently no ideal gold standard for assessing the prognosis of patients with acute stroke. The preliminary study conducted by the investigators revealed that the order of clinically recommended prognostic scoring methods for stroke was as follows: APACHE II, APACHE III, NIHSS, ADL, CSS, and GCS. APACHE II is preferentially recommended.

Data Availability Statement

The datasets generated for this study are available on request to the corresponding author.

Ethics Statement

This study was carried out in accordance with the recommendations of the declaration of Helsinki and the ethical committee of Affiliated Hospital of North China University of Science and Technology, with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the ethical committee of Affiliated Hospital of North China University of Science and Technology.

Author Contributions

Q-XL been involved in drafting the work and revising it critically for important intellectual content. X-JZ given final approval of the version to be published. H-YF was involved in article revision. X-NL and X-JW made substantial contributions to the conception and design of the work. JZ, D-LW, R-YC, and LZ made substantial contributions to the acquisition, analysis, and interpretation of data for the work.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

1. Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med. (1985) 13:818–29. doi: 10.1097/00003246-198510000-00009

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Knaus WA, Wagner DP, Draper EA, Zimmerman JE, Bergner M, Bastos PG, et al. The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest. (1991) 100:1619–36. doi: 10.1378/chest.100.6.1619

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Chen QT. [Evaluation criteria of clinical neurological impairment degree in stroke patients]. Chin J Neurol. (1996) 35:62–4. (Article in Chinese)

4. Goldstein LB, Bertels C, Davis JN. Interrater reliability of the NIH stroke scale. Arch Neurol. (1989) 46:660–2. doi: 10.1001/archneur.1989.00520420080026

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Mahoney FI, Barthel DW. Functional evaluation: the barthel index. Md State Med J. (1965) 14:61–5. doi: 10.1037/t02366-000

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Jennett B, Bond M. Assessment of outcome after severe brain damage. Lancet. (1975) 1:480–4. doi: 10.1016/S0140-6736(75)92830-5

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Hanley J, McNeil B. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology. (1983) 148 839–43. doi: 10.1148/radiology.148.3.6878708

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Sakallaris BR, Jastremski CA, Von Rueden KT. Clinical decision support systems for outcome measurement and management. AACN Clin Issues. (2000) 11:351–62. doi: 10.1097/00044067-200008000-00003

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Tao ZR. [Evaluation of reliability, validity and sensitivity of clinical neurological impairment scoring criteria in stroke patients in China]. Acad J Sec Mil Med Univ. (2009) 30:283–5. (Article in Chinese). doi: 10.3724/SP.J.1008.2009.00283

CrossRef Full Text | Google Scholar

10. Murthy JM, Meena AK, Kumar SR. Severity-of-illness scoring systems and models: neurological and neurosurgical intensive care units. Neurol Ind. (2001) 49 (Suppl. 1):S91–4.

PubMed Abstract | Google Scholar

11. Le Gall JR. [The performance of intensive care services]. Bull Acad Natl Med. (2000) 184:1653–63. (Article in French).

PubMed Abstract | Google Scholar

12. Jiang XC. [Clinical application and significance of severity score of critical disease]. Chin Crit Care Med. (2000) 34:195–7. (Article in Chinese) doi: 10.3760/j.issn:1003-0603.2000.04.001

CrossRef Full Text

13. Candelise L, Bogousslavsky J, Hennerici MG. Stroke scores and scales. Cerebrovasc Dis. (1992) 2:239–47. doi: 10.1159/000109022

CrossRef Full Text | Google Scholar

14. Ding JR, Guan YX, Wu DM, Jiang SL. [Application of APACHE II score in neurosurgery intensive ward]. J Qiqihar Med Univ. (2016) 37:730–1. (Article in Chinese)

15. Wang SH, He GP, Zeng C, Zhou JH, Xia XC. [Evaluation of APACHE II score on the condition and prognosis of patients undergoing cardiac macrovascular surgery]. J Cent S Univ. (2013) 38:419–24. (Article in Chinese) doi: 10.3969/j.issn.1672-7347.2013.04.014

CrossRef Full Text

16. Sui H, Chen WW, Wang W. [Introduction to the key points of China cardiovascular disease report 2014]. Chin J Hypertens. (2015) 23:627–9. (Article in Chinese). doi: 10.16439/j.cnki.1673-7245.2015.07.012

CrossRef Full Text

17. Wang YM, Wei TT, Hou M, Zhang L, Aziguli Memeti, Li P. [A comparative study on the evaluation of prognosis ability of emergency internal medicine patients by using improved early warning score and APACHE II]. Chin Nurs Res. (2016) 30:1562–6. (Article in Chinese) doi: 10.3969/j.issn.1009-6493.2016.013.008

CrossRef Full Text

18. Han HG, Wang HS, Li XM, Xu LY, Meng QT, Zhang XX, et al. [Evaluation of the prognostic value of platelet detection combined with APACHE II scoring system in cardiac surgery intensive care unit]. Clin Misdiag Misther. (2016) 29:67–70. (Article in Chinese) doi: 10.3969/j.issn.1002-3429.2016.07.025

CrossRef Full Text

19. Cooray C, Fekete K, Mikulik R, Lees KR, Wahlgren N, Ahmed N. Threshold for NIH stroke scale in predicting vessel occlusion and functional outcome after stroke thrombolysis. Int J Stroke. (2015) 10:822–9. doi: 10.1111/ijs.12451

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Yaghi S, Herber C, Boehme AK, Andrews H, Willey JZ, Rostanski SK, et al. The association between diffusion MRI-defined infarct volume and NIHSS score in patients with minor acute stroke. J Neuroimaging. (2017) 27:388–91. doi: 10.1111/jon.12423

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Shrestha S, Poudel RS, Khatiwada D, Thapa L. Stroke subtype, age, and baseline NIHSS score predict ischemic stroke outcomes at 3 months: a preliminary study from Central Nepal. J Multidiscip Healthc. (2015) 8:443–8. doi: 10.2147/JMDH.S90554

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Chang WH, Sohn MK, Lee J, Kim DY, Lee SG, Shin YI, et al. Long-term functional outcomes of patients with very mild stroke: does a NIHSS score of 0 mean no disability? An interim analysis of the KOSCO study. Disabil Rehabil. (2017) 39:904–10. doi: 10.3109/09638288.2016.1170214

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Chen HB, Zhan Y, Li SF, Tong LW, Zheng T. [Comparative study on the effect of APACHE II/III scoring system on prognosis evaluation of hospitalized patients in general hospital]. China J Mod Med. (2016) 26:115–9. (Article in Chinese) doi: 10.3969/j.issn.1005-8982.2016.10.025

CrossRef Full Text

24. Liu X, Chen ZY, Li RX, Liu J, Yang CS. [Application value of modified APACHE II scoring system in prognosis evaluation of chronic alcoholic toxic encephalopathy]. World Latest Med Inform. (2016) 16:13–4. (Article in Chinese)

25. Su J. [Reliability and validity of clinical neurological impairment degree score in stroke patients]. Guide China Med. (2013) 11:626–7. (Article in Chinese) doi: 10.3969/j.issn.1671-8194.2013.17.480

CrossRef Full Text

26. Zhao XJ, Li QX, Liu TJ, Wang DL, An YC, Zhang J, et al. Predictive values of CSS and NIHSS in the prognosis of patients with acute cerebral infarction: a comparative analysis. Medicine. (2018) 97:e12419. doi: 10.1097/MD.0000000000012419

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Teasdale G, Jennett B. Assessment of coma and impaired consciousness. a practical scale. Lancet. (1974) 2:81–4. doi: 10.1016/S0140-6736(74)91639-0

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Teasdale G, Jennett B. Assessment and prognosis of coma after head injury. Acta Neurochir. (1976) 34:45–55. doi: 10.1007/BF01405862

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Nan DK, Liao HS. Rehabilitation Medicine, 1st Ed. Beijing: People's Medical Publishing House (1993).

30. Chen L, Wang YC. [Prediction of prognosis of stroke patients by APACHE II score and comparison with GCS score]. Chin J Nerv Ment Dis. (1997) 23:216–8. (Article in Chinese)

31. Bein T, Fröhlich D, Frey A, Metz C, Taeger K. [Comparison of APACHE-II and APACHE-III for classification of disease severity of intensive care patients]. Anaesthesist. (1995) 44:37–42. (Article in German). doi: 10.1007/s001010050130

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Wang XP. [Clinical significance of APACHE III, SAPS II and APACHE II scoring system in evaluating the condition of critically ill patients in emergency internal medicine]. Chin Foreign Med Res. (2015) 13:133–4. (Article in Chinese) doi: 10.14033/j.cnki.cfmr.2015.31.065

CrossRef Full Text

33. Yu XJ, Tang XJ, Sun X. [Study on the application of APACHE III scoring system in evaluating the condition and prognosis of stroke]. Cardiovasc Dis J Integr Tradit Chin West Med. (2018) 6:60–2. (Article in Chinese) doi: 10.3969/j.issn.2095-6681.2018.12.044

CrossRef Full Text