An Externally Validated Dynamic Nomogram for Predicting Unfavorable Prognosis in Patients With Aneurysmal Subarachnoid Hemorrhage

Background: Aneurysmal subarachnoid hemorrhage (aSAH) leads to severe disability and functional dependence. However, no reliable method exists to predict the clinical prognosis after aSAH. Thus, this study aimed to develop a web-based dynamic nomogram to precisely evaluate the risk of poor outcomes in patients with aSAH.

Methods: Clinical patient data were retrospectively analyzed at two medical centers. One center with 126 patients was used to develop the model. Least absolute shrinkage and selection operator (LASSO) analysis was used to select the optimal variables. Multivariable logistic regression was applied to identify independent prognostic factors and construct a nomogram based on the selected variables. The C-index and Hosmer–Lemeshow p-value and Brier score was used to reflect the discrimination and calibration capacities of the model. Receiver operating characteristic curve and calibration curve (1,000 bootstrap resamples) were generated for internal validation, while another center with 84 patients was used to validate the model externally. Decision curve analysis (DCA) and clinical impact curves (CICs) were used to evaluate the clinical usefulness of the nomogram.

Results: Unfavorable prognosis was observed in 46 (37%) patients in the training cohort and 24 (29%) patients in the external validation cohort. The independent prognostic factors of the nomogram, including neutrophil-to-lymphocyte ratio (NLR) (p = 0.005), World Federation of Neurosurgical Societies (WFNS) grade (p = 0.002), and delayed cerebral ischemia (DCI) (p = 0.0003), were identified using LASSO and multivariable logistic regression. A dynamic nomogram (https://hu-ping.shinyapps.io/DynNomapp/) was developed. The nomogram model demonstrated excellent discrimination, with a bias-corrected C-index of 0.85, and calibration capacities (Hosmer–Lemeshow p-value, 0.412; Brier score, 0.12) in the training cohort. Application of the model to the external validation cohort yielded a C-index of 0.84 and a Brier score of 0.13. Both DCA and CIC showed a superior overall net benefit over the entire range of threshold probabilities.

Conclusion: This study identified that NLR on admission, WFNS grade, and DCI independently predicted unfavorable prognosis in patients with aSAH. These factors were used to develop a web-based dynamic nomogram application to calculate the precise probability of a poor patient outcome. This tool will benefit personalized treatment and patient management and help neurosurgeons make better clinical decisions.

Introduction

Aneurysmal subarachnoid hemorrhage (aSAH) is an acute cerebrovascular injury with high morbidity and mortality worldwide (1). Roughly one-third of survivors have severe disability and functional dependence, some of which may be predicted and treated early (2, 3). Therefore, a practical prediction model for treating and evaluating these patients with aSAH is urgently needed.

Several studies have shown that a series of serum biochemical and immunological indices could predict the prognosis of patients with aSAH after surgery, including white blood cell (WBC) count, neutrophil-to-lymphocyte ratio (NLR), C-reactive protein level, and blood glucose level on admission (4–6). Meanwhile, predictive models combining multiple independent predictors have been used to predict prognosis. However, these predictive models lack calibration tests, model external validation, or evaluation of their clinical usefulness; therefore, the accuracy and practicability of the models are difficult to assess (7, 8). High-quality predictive models are necessary to guide clinical decision-making, personal care, and patient management (9). Therefore, a more practical and accurate predictive model is urgently needed in this field. A web-based dynamic nomogram that precisely calculates the probability of the disease is a more precise and practical tool than standard nomograms and some predictive models (10, 11). However, no study has reported a dynamic nomogram to predict the clinical prognosis after aSAH.

Thus, this study aimed to develop a dynamic web-based nomogram that incorporated blood laboratory tests, clinical state at admission, and postoperative complications for the prognosis of patients with aSAH. The predictive model was validated using internal and independent external validation cohorts.

Materials and Methods

Study Design and Patient Enrollment

This retrospective observational cohort study was conducted in two centers. A training cohort of 126 patients with aSAH treated in the Department of Neurosurgery, Renmin Hospital of Wuhan University, from September 1, 2019 to September 1, 2020, was used to develop the model. A total of 84 patients admitted to the Department of Neurosurgery, the Affiliated Hospital of Panzhihua University, from September 1, 2019 to September 1, 2020, were used to validate the model. The diagnosis of aSAH was assessed by head CT, CT angiography, or digital subtraction angiography according to the guidelines (12, 13).

The exclusion criteria were as follows: (1) non-aneurysmal or traumatic SAH; (2) clinical history of aSAH for more than 48 h; (3) complicated with vascular anomalies and malformations; (4) complicated with intracerebral hemorrhage; (5) taking anticoagulants or corticosteroids within 1 month of hospitalization; (6) postoperative state on admission; (7) no surgical treatment performed within 2 days of onset; (8) acute infection on admission; (9) other organs with extreme dysfunction; (10) bilateral mydriasis or other permanent brain injuries on admission; (11) missing data.

Data Collection

Clinical information including demographic data (admission number, name, sex, age, hypertension, smoking, and alcohol consumption), clinical state, radiological findings, serum laboratory tests (glucose, D-dimer, WBCs, neutrophils, lymphocytes, and monocytes), and CT Hounsfield unit (HU) values on admission were collected using an electronic medical record system. The number, location length, and neck size of the aneurysms were also recorded. Hunt and Hess classification and the World Federation of Neurosurgical Societies (WFNS) were used to measure the clinical characteristics and neurological status at admission. The radiological features at admission were classified according to the modified Fisher grade of the CT scan (14–16).

All patients were treated with either a surgical clipping or coil embolization as soon as possible. The postoperative routine therapies included hemostasis, analgesics, anti-inflammatory drugs, and nimodipine for anti-vasospasm. An immediate postoperative head CT scan was performed to identify intracranial rebleeding or cerebral infarction after the operation.

Admission CT Values

The mean CT values of the blood clots in the subarachnoid space were measured. The regions of interest were manually drawn on representative slices by two neurosurgeons who were blinded to the patients' clinical characteristics. The density of blood clotting in the subarachnoid space on CT slices was assessed using the mean HU value. The following subarachnoid cisterns/fissures were used to measure the mean HU: lateral Sylvian fissures, anterior interhemispheric fissures, medial Sylvian fissures, suprasellar cistern, ambient cistern, and quadrigeminal cistern, as previously described (17, 18).

The Definition of Delayed Cerebral Ischemia

Delayed cerebral ischemia (DCI) was identified as follows: (1) CT scan omitting other causes of focal neurological decline such as aphasia, apraxia, hemianopia, or neglect, either permanent or temporary, within 4 to 14 days after SAH; (2) a Glasgow Coma Scale decrease of at least 2 points lasting for at least 1 h and not immediately evident after surgery; (3) head CT scan revealing a new cerebral infarction within 4–30 days after aSAH, which was not noticeable on admission or immediately after the operation, and no other explanation except for vasospasm (17, 19).

Outcome Assessment

Three months after the initial bleeding, the functional outcome was dichotomized as excellent or poor according to the modified Rankin Scale (mRS) (20–22). A favorable outcome was defined as a score of 0–2 (perform activities without assistance) on the mRS, while a score of 3–6 (moderate to severe disability or death) was defined as a poor outcome. The results were evaluated via telephone by a neurosurgeon who was blinded to the clinical and imaging data.

Sample Size

In the current literature, the events per variable (EPV) criterion, notably an EPV of 10, is widely applied as the lower limit for developing logistic regression models that predict a binary outcome (23, 24). Thus, an EPV of 10 was used to estimate the effective sample size in this study. A total of three variables, which were incorporated into the multivariate logistic analysis, were selected by least absolute shrinkage and selection operator (LASSO) regression analysis. Thus, the effective sample size was at least 30 in the training cohort.

Statistical Analysis

Statistical analysis was performed using IBM SPSS Statistics for Windows, version 26.0 (IBM Corp., Armonk, NY, USA) and R software (https://www.r-project.org/). Continuous variables were analyzed using independent t-test or Mann–Whitney U test and are presented as means ± SD or medians with interquartile range. Categorical variables are expressed as numbers (percentages) and were analyzed using χ² or Fisher exact tests. Based on the limited sample size in this study, LASSO regression, which is suitable for analyzing high-dimensional data, was used to select the most informative prognostic variables from the data cohort (25–27). The variables were then entered into a multivariable logistic regression model; a nomogram model was then constructed by integrating independent factors.

The discrimination and calibration capacities evaluated the performance of the nomogram model. Receiver operating characteristic (ROC) curve and area under the curve (AUC) analyses were performed to assess the discriminatory capabilities, while the bias-corrected C-index was calculated using 1,000 bootstraps. The Hosmer–Lemeshow test, Brier score, and calibration curve were used to assess the calibration ability (28). In addition to internal validation, the model performance was also assessed using an external validation cohort. The clinical effectiveness of the nomogram was evaluated by decision curve analysis (DCA) and clinical impact curve (CIC) (29, 30).

We used the “glmnet, corrplot, caret package” in R software to generate the LASSO regression results and the “rms package” to establish the nomogram. The “pROC,” “rmda,” “MASS,” “survival,” “ggplot2,” “ggridges,” “plotROC,” and “riskRegression” packages were applied to generate the C-index, ROC, DCA, CIC, and calibration curve. The “shinyapps.io” and “DynNom packages” were used to develop a web-based dynamic nomogram application to predict unfavorable prognosis, which could precisely calculate the risk probability for unfavorable prognosis at 3 months after aSAH. All tests were two-tailed, and p-values < 0.05 were considered statistically significant.

Results

Baseline Cohort Characteristics

This study included a total of 210 patients with aSAH, with 126 and 84 patients in the training and external validation cohorts, respectively. The baseline characteristics of the two cohorts were shown in Table 1. Women comprised 137 (65%) patients in the two cohorts, and the mean age was 56.50 years (interquartile range: 53.00, 63.00). We observed no significant differences between the training and external validation cohorts, except for higher admission glucose level, percentages of DCI occurrence and WFNS grade IV–V, and lower aneurysm neck size in the training cohort. The numbers of patients with poor prognoses in the two cohorts, 46 (37%) and 24 patients (29%), respectively, did not differ significantly. Table 2 showed the baseline characteristics of the model development cohort.

TABLE 1

Table 1. Baseline characteristics of the training and external validation cohorts.

TABLE 2

Table 2. Baseline characteristics of the training cohort according to the prognostic outcomes 3 months after aSAH.

Variable Selection

LASSO regression was used to select the most useful variables. As shown in Figure 1A, when using the minimum error criterion, 21 variables were decreased to 11 variables and further reduced to three variables after applying the one standard error (1-SE) criterion. Thus, an optimal λ of 0.1356, with log(λ) = −1.997, was adopted. Three factors with non-zero coefficients were finally selected by 5-fold cross-validation to prevent overfitting (Figure 1B). After adjusting by multivariable logistic regression, NLR [adjusted odds ratio (aOR), 1.143; 95% CI 1.045–1.263, p = 0.005], WFNS grade (aOR: 5.025, 95% CI 1.777–14.75, p = 0.002), and DCI (aOR: 6.143, 95% CI 2.374–18.13, p = 0.0003) were independent prognostic factors (Table 3).

FIGURE 1

Figure 1. Selection of prognostic variables by least absolute shrinkage and selection operator (LASSO) regression analysis. (A) The selection of optimal parameters (lambda) by 5-fold cross-validation. The left and right dotted vertical lines, respectively, represent the optimal lambda values when using the minimum error criterion and one standard error (1-SE) of the minimum criterion. (B) The vertical line was plotted at the optimal λ of 0.1356, with log(λ) = −1.997. Three factors with non-zero coefficients were finally selected.

TABLE 3

Table 3. Multivariable logistic analysis for variables selected by LASSO.

Prognostic Model Development

All independent factors were used to construct a nomogram to predict unfavorable prognosis 3 months after aSAH (Figure 2). According to the patient's information on admission, each of the three prognostic factors in the nomogram was projected upward to a point. NLR had continuous values ranging from 0 to 100. The WFNS grade was divided into two levels (I–III and IV–V), while DCI was divided into non-DCI and DCI, each of which were assigned points. The total sum of the points from the three variables was converted into an individual poor prognosis risk, in which the higher was the total score, the higher was the risk of an unfavorable prognosis. Based on the ordinary nomogram, we developed a dynamic nomogram web-based application (https://hu-ping.shinyapps.io/DynNomapp/) to precisely calculate the risk probability for unfavorable prognosis at 3 months after aSAH.

FIGURE 2

Figure 2. The ordinary nomogram predicts the risk of unfavorable prognosis 3 months after aSAH, based on NLR, WFNS grade, and DCI. NLR, neutrophil-to-lymphocyte ratio; WFNS, World Federation of Neurosurgical Societies; DCI, delayed cerebral ischemia.

Nomogram Validation

The original AUC was 0.88 (95% CI 0.82–0.95), while the bias-corrected C-index with 1,000 bootstraps was 0.82, suggesting that the model had excellent discrimination. Compared to a single independent prognostic factor, the nomogram model better predicted the occurrence of DCI (Figure 3A). The calibration capacity was also internally validated; the Hosmer–Lemeshow p-value of 0.41 in the training cohort suggested an excellent fitting of the nomogram. By contrast, a Brier score of 0.12 and a calibration curve with 1,000 bootstrap resamples showed that the model has a good calibration ability (Figure 4A), demonstrating no significant deviation between the actual and predicted probabilities.

FIGURE 3

Figure 3. ROC and AUC analysis for nomogram validation. (A) Internal validation. (B) External validation. AUC, area under the curve; ROC, receiver operating characteristic; NLR, neutrophil-to-lymphocyte ratio; WFNS, World Federation of Neurosurgical Societies; DCI, delayed cerebral ischemia.

FIGURE 4

Figure 4. Calibration curve for nomogram validation. (A) Internal validation. (B) External validation.

A total of 84 patients retrospectively collected from September 2019 to September 2020 at another center were used for external validation of the model. The validation AUC value of 0.84 (95% CI 0.74–0.94) was consistent with the original AUC value (0.88) in the training cohort (Figure 3B). A Brier score of 0.13 and the calibration curve plotted for external validation showed good calibration of the model in the external validation cohort (Figure 4B). Furthermore, DCA and CIC were used to evaluate the clinical usefulness of our prognostic nomogram. DCA showed a superior overall net benefit for a threshold probability of 0–1 (Figure 5A). CIC also demonstrated good performance over the entire range of threshold probabilities (Figure 5B).

FIGURE 5

Figure 5. DCA and CIC curves. (A) DCA curve. (B) CIC curve. DCA, decision curve analysis; CIC, clinical impact curve.

Discussion

This study constructed and validated a dynamic nomogram model to predict unfavorable prognosis at 3 months after aSAH. The nomogram incorporated the NLR, admission WFNS grade, and DCI. To our knowledge, this study is the first to develop a web-based dynamic nomogram application to calculate the risk of poor prognosis (mRS, 3–6 points). In addition, external validation using data from another center was conducted for the first time. Precise calculation of the probability of the disease makes the dynamic nomogram a more accurate and practical tool compared to ordinary predictive models.

In a previous study, Zhang et al. developed an ordinary nomogram combined with a systemic inflammation response index and other conventional factors to predict the prognostic outcome in patients with aSAH (8). These factors were selected using univariate and multivariate logistic analyses. Lai et al. conducted a predicting model for aSAH prognosis, which included the GCS score, surgical clipping, and NLR (7). These independent prognostic variables were also selected by multivariate logistic regression. Compared with the study by Zhang et al., the model by Lai et al. was only internally validated rather than externally validated. Although a large-scale retrospective study has been conducted on poor recovery in patients with aSAH, the study only contained an internal validation (31). Our study included a total of 84 patients retrospectively collected from September 2019 to September 2020 from another center for external validation of the model. The validation AUC value was nearly identical to the original AUC value in the training cohort. The calibration curve showed good calibration of the model in the external validation cohort.

Contrary to the conventional multivariate logistic method, this study applied the LASSO regression analysis, which performed well in reducing the data dimensionality and decreasing multicollinearity between variables, and it was used to select variables for multivariable logistic regression by minimizing coefficients and reducing variance (32). As a result, an optimal λ of 0.1356, with log(λ) = −1.997, was adapted. Three factors with non-zero coefficients were finally selected by 5-fold cross-validation to prevent overfitting. A logistic regression model was then used to establish a nomogram. The original AUC was 0.88, while the bias-corrected C-index with 1,000 bootstraps was 0.82, suggesting that the model had excellent discrimination. Compared with a single independent prognostic factor, the nomogram model better predicted the occurrence of DCI. The calibration capacity was also internally validated; the Hosmer–Lemeshow p-value in the training cohort suggested an excellent fit of the nomogram. By contrast, a Brier score and calibration curve with 1,000 bootstrap resamples showed that the model had a good calibration ability, demonstrating no significant deviation between the actual and predicted probabilities. In addition, DCA showed a superior overall net benefit for a threshold probability of 0–1. The CIC also demonstrated a good performance over the entire range of threshold probabilities.

We observed no significant difference in NLR between the training and external validation cohorts. However, the NLR in patients with an unfavorable prognosis was higher than that in patients with a favorable prognosis in the training cohort (p = 0.005). Our study identified NLR on admission as an independent risk factor for unfavorable prognosis, similar to previous findings (4, 7, 33, 34). Lai et al. reported that a high NLR on admission could increase the risk of poor prognosis 3 months postoperatively. Giede-Jeppe et al. suggested that higher NLR was associated with more significant neurological injury, as reflected by higher subarachnoid and intraventricular blood. Therefore, increased NLR resulted in worse functional outcomes beyond these associations. The following possible mechanisms may explain why high NLR on admission increased the risk of unfavorable prognosis at 3 months after aSAH. First, a high NLR could promote the synthesis and secretion of matrix metalloproteinase, which could degrade components of the extracellular matrix and play an essential role in the blood–brain barrier, resulting in secondary brain injury (35, 36). Second, increased NLR at admission may include excessive neuroinflammation caused by elevated neutrophil granulocyte counts and lymphocyte depletion leading to immunodepression. Physiological stress responses lead to an increased postictal release of endogenous catecholamines and cortisol, resulting in secondary brain injury and possibly inflammatory complications (37, 38).

The WFNS grade or GCS have gradually become essential validated scales to evaluate clinical characteristics and neurological status on admission. A good grade was defined as I–III on WFNS, while IV–V on the WFNS was considered poor (39, 40). In our study, 40 (32%) and 11 (13%) patients in the training and external validation cohorts, respectively, had WFNS grades of IV–V. WFNS grade (p = 0.002) on admission independently predicted unfavorable prognosis and patients with WFNS grades IV–V had a higher risk of developing a poor prognosis compared with patients with WFNS grades I–III. This finding is similar to those of previous studies showing that WFNS grades IV–V was correlated with a poor outcome and was a good predictor of unfavorable prognosis after aSAH (41, 42).

Furthermore, the DCI occurrence in the two cohorts was 35 and 15%, respectively. The onset of aSAH was concentrated during the COVID-19 period, which resulted in patients not receiving timely anti-vasospasm. It may be interpreted that the training cohort had a larger percentage of DCI. DCI was identified as another crucial prognostic factor in this study. The results of a large-scale retrospective study also suggested that DCI plays an important role in short-term and long-term prognosis (43). Cerebrovascular disorders, microthrombosis, cortical diffuse depolarization, and neuroinflammation all reportedly play potential roles in DCI pathogenesis (17, 44–46).

Contrary to nomograms or other models calculated as an approximation in previous studies, the web-based dynamic nomogram application developed in the present study (https://hu-ping.shinyapps.io/DynNomapp/) can be used to calculate an exact value for developing poor outcomes at 3 months after aSAH. This tool will help in the individualized treatment and management of patients, including neuro-intensive care, blood pressure control, nimodipine anti-vasospasm, anti-inflammatory drugs, and rehabilitation treatment, and help neurosurgeons make better clinical decisions.

However, this study has several limitations. First, this was a retrospective study; therefore, prospective studies are needed to verify our results. Second, owing to the low effective sample size in the training cohort, to reduce the data dimensionality and decrease multicollinearity between variables, we used LASSO regression analysis to select predictors. Third, future studies should collect complete clinical treatment information.

Conclusion

This study identified independent predictors for an unfavorable prognosis in patients with aSAH, including NLR on admission, the WFNS grade, and DCI. On this basis, a web-based dynamic nomogram application, after externally validated, was developed to calculate the precise probability of a poor outcome in patients 3 months after aSAH. This will benefit personalized treatment and patient management and help neurosurgeons make better clinical decisions.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by the Medical Ethics Committee of Renmin Hospital of Wuhan University. (approval number WDRM2021-K022). Written informed consent from the patients/participants or patients/participants legal guardian/next of kin was not required to participate in this study in accordance with the national legislation and the institutional requirements.

Author Contributions

PH, YX, GD, and QC: Study design. PH, SZ, XZ, QS, and YW: Literature search. PH, YLiu, YLi, and LY: Data acquisition, data analysis, and statistical analysis. PH, YX, YQ, HZ, GD, and QC: Article preparation, editing, and review. All authors read and approved the final article.

Funding

This work was supported by the National Natural Science Foundation of China (No. 82001311).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Connolly ES Jr, Rabinstein AA, Carhuapoma JR, Derdeyn CP, Dion J, Higashida RT, et al. Guidelines for the management of aneurysmal subarachnoid hemorrhage: a guideline for healthcare professionals from the American Heart Association/american Stroke Association. Stroke. (2012) 43:1711–37. doi: 10.1161/STR.0b013e3182587839

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Rinkel GJ, Algra A. Long-term outcomes of patients with aneurysmal subarachnoid haemorrhage. Lancet Neurol. (2011) 10:349–56. doi: 10.1016/S1474-4422(11)70017-5

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Mackey J, Khoury JC, Alwell K, Moomaw CJ, Kissela BM, Flaherty ML, et al. Stable incidence but declining case-fatality rates of subarachnoid hemorrhage in a population. Neurology. (2016) 87:2192–7. doi: 10.1212/WNL.0000000000003353

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Giede-Jeppe A, Reichl J, Sprügel MI, Lücking H, Hoelter P, Eyüpoglu IY, et al. Neutrophil-to-lymphocyte ratio as an independent predictor for unfavorable functional outcome in aneurysmal subarachnoid hemorrhage. J Neurosurg. (2019) 132:400–7. doi: 10.3171/2018.9.JNS181975

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Zhang D, Zhuang Z, Wei Y, Liu X, Li W, Gao Y, et al. Association of admission serum glucose-phosphate ratio with severity and prognosis of aneurysmal subarachnoid hemorrhage. World Neurosurg. (2019) 127:e1145–51. doi: 10.1016/j.wneu.2019.04.071

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Ma X, Lan F, Zhang Y. Associations between C-reactive protein and white blood cell count, occurrence of delayed cerebral ischemia and poor outcome following aneurysmal subarachnoid hemorrhage: a systematic review and meta-analysis. Acta Neurol Belg. (2021) 9:1–14. doi: 10.1007/s13760-020-01496-y

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Lai X, Zhang W, Ye M, Liu X, Luo X. Development and validation of a predictive model for the prognosis in aneurysmal subarachnoid hemorrhage. J Clin Lab Anal. (2020) 34:e23542. doi: 10.1002/jcla.23542

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Zhang P, Li Y, Zhang H, Wang X, Dong L, Yan Z, et al. Prognostic value of the systemic inflammation response index in patients with aneurismal subarachnoid hemorrhage and a Nomogram model construction. Br J Neurosurg. (2020) 12:1–7. doi: 10.1080/02688697.2020.1831438

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Jaja BN, Cusimano MD, Etminan N, Hanggi D, Hasan D, Ilodigwe D, et al. Clinical prediction models for aneurysmal subarachnoid hemorrhage: a systematic review. Neurocrit Care. (2013) 18:143–53. doi: 10.1007/s12028-012-9792-z

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Chen S, Li X, Lv H, Wen X, Ding Q, Xue N, et al. Prognostic dynamic nomogram integrated with inflammation-based factors for non-small cell lung cancer patients with chronic hepatitis B viral infection. Int J Biol Sci. (2018) 14:1813–21. doi: 10.7150/ijbs.27260

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Chen L, Qian J, Lin L, Lin J, Chen Q, Zhuang Z, et al. Prognostic value of preoperative lymphocyte-to-monocyte ratio in oral cancer patients and establishment of a dynamic nomogram. Oral Dis. (2021) 27:1127–36. doi: 10.1111/odi.13629

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Bederson JB, Connolly ES Jr, Batjer HH, Dacey RG, Dion JE, Diringer MN, et al. Guidelines for the management of aneurysmal subarachnoid hemorrhage: a statement for healthcare professionals from a special writing group of the Stroke Council, American Heart Association. Stroke. (2009) 40:994–1025. doi: 10.1161/STROKEAHA.108.191395

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Ajiboye N, Chalouhi N, Starke RM, Zanaty M, Bell R. Unruptured cerebral aneurysms: evaluation and management. Sci World J. (2015) 2015:954954. doi: 10.1155/2015/954954

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Frontera JA, Claassen J, Schmidt JM, Wartenberg KE, Temes R, Connolly ES Jr, et al. Prediction of symptomatic vasospasm after subarachnoid hemorrhage: the modified fisher scale. Neurosurgery. (2006) 59:21–7; discussion 21–7. doi: 10.1227/01.NEU.0000218821.34014.1B

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Sano H, Satoh A, Murayama Y, Kato Y, Origasa H, Inamasu J, et al. Modified World Federation of Neurosurgical Societies subarachnoid hemorrhage grading system. World Neurosurg. (2015) 83:801–7. doi: 10.1016/j.wneu.2014.12.032

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Zhang Y, Zhu X, Hou K, Zhao J, Gao X, Sun Y, et al. Clinical outcomes of surgical clipping for intracranial aneurysms in patients with a Hunt and Hess grade 4 or 5. Arq Neuropsiquiatr. (2016) 74:478–81. doi: 10.1590/0004-282x20160064

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Vergouwen MD, Vermeulen M, Coert BA, Stroes ES, Roos YB. Microthrombosis after aneurysmal subarachnoid hemorrhage: an additional explanation for delayed cerebral ischemia. J Cereb Blood Flow Metab. (2008) 28:1761–70. doi: 10.1038/jcbfm.2008.74

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Woo PYM, Tse TPK, Chan RSK, Leung LNY, Liu SKK, Leung AYT, et al. Computed tomography interobserver agreement in the assessment of aneurysmal subarachnoid hemorrhage and predictors for clinical outcome. J Neurointerv Surg. (2017) 9:1118–24. doi: 10.1136/neurintsurg-2016-012576

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Vergouwen MD, Vermeulen M, Van Gijn J, Rinkel GJ, Wijdicks EF, Muizelaar JP, et al. Definition of delayed cerebral ischemia after aneurysmal subarachnoid hemorrhage as an outcome event in clinical trials and observational studies: proposal of a multidisciplinary research group. Stroke. (2010) 41:2391–5. doi: 10.1161/STROKEAHA.110.589275

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Newcommon NJ, Green TL, Haley E, Cooke T, Hill MD. Improving the assessment of outcomes in stroke: use of a structured interview to assign grades on the modified Rankin Scale. Stroke. (2003) 34:377–8; author reply 377–8. doi: 10.1161/01.STR.0000055766.99908.58

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Banks JL, Marotta CA. Outcomes validity and reliability of the modified Rankin scale: implications for stroke clinical trials: a literature review and synthesis. Stroke. (2007) 38:1091–6. doi: 10.1161/01.STR.0000258355.23810.c6

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Janssen PM, Visser NA, Dorhout Mees SM, Klijn CJ, Algra A, Rinkel GJ. Comparison of telephone and face-to-face assessment of the modified Rankin Scale. Cerebrovasc Dis. (2010) 29:137–9. doi: 10.1159/000262309

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Moons KG, De Groot JA, Bouwmeester W, Vergouwe Y, Mallett S, Altman DG, et al. Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist. PLoS Med. (2014) 11:e1001744. doi: 10.1371/journal.pmed.1001744

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Pavlou M, Ambler G, Seaman S, De Iorio M, Omar RZ. Review and evaluation of penalised regression methods for risk prediction in low-dimensional data with few events. Stat Med. (2016) 35:1159–77. doi: 10.1002/sim.6782

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Steyerberg EW, Eijkemans MJ, Harrell FE Jr, Habbema JD. Prognostic modeling with logistic regression analysis: in search of a sensible strategy in small data sets. Med Decis Making. (2001) 21:45–56. doi: 10.1177/0272989X0102100106

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Sauerbrei W, Royston P, Binder H. Selection of important variables and determination of functional form for continuous predictors in multivariable model building. Stat Med. (2007) 26:5512–28. doi: 10.1002/sim.3148

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Riley RD, Snell KIE, Martin GP, Whittle R, Archer L, Sperrin M, et al. Penalization and shrinkage methods produced unreliable clinical prediction models especially when sample size was small. J Clin Epidemiol. (2020) 132:88–96. doi: 10.1016/j.jclinepi.2020.12.005

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Hanley JA, Mcneil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. (1982) 143:29–36. doi: 10.1148/radiology.143.1.7063747

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. (2006) 26:565–74. doi: 10.1177/0272989X06295361

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Van Calster B, Wynants L, Verbeek JFM, Verbakel JY, Christodoulou E, Vickers AJ, et al. Reporting and interpreting decision curve analysis: a guide for investigators. Eur Urol. (2018) 74:796–804. doi: 10.1016/j.eururo.2018.08.038

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Yan Y, Hu J, Fang X, Zhen Y, Feng L, Zhang X, et al. Predicting the poor recovery risk of aneurysmal subarachnoid hemorrhage: clinical evaluation and management based on a new predictive nomogram. Clin Neurol Neurosurg. (2021) 200:106302. doi: 10.1016/j.clineuro.2020.106302

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Akkol S. The prediction of live weight of hair goats through penalized regression methods: LASSO and adaptive LASSO. Arch Anim Breed. (2018) 61:451–8. doi: 10.5194/aab-61-451-2018

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Guo R, Wu Y, Chen R, Yu Z, You C, Ma L, et al. Clinical value of neutrophil-to-lymphocyte ratio in primary intraventricular hemorrhage. World Neurosurg. (2019) 127:e1051–6. doi: 10.1016/j.wneu.2019.04.040

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Wang JY, Zhang XT, Wang JQ, Wang CY, Zheng WL, Pan ZM, et al. Admission neutrophil-lymphocyte ratio predicts rebleeding following aneurismal subarachnoid hemorrhage. World Neurosurg. (2020) 138:e317–22. doi: 10.1016/j.wneu.2020.02.112

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Lazaridis C, Rusin CG, Robertson CS. Secondary brain injury: predicting and preventing insults. Neuropharmacology. (2019) 145:145–52. doi: 10.1016/j.neuropharm.2018.06.005

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Lattanzi S, Di Napoli M, Ricci S, Divani AA. Matrix metalloproteinases in acute intracerebral hemorrhage. Neurotherapeutics. (2020) 17:484–96. doi: 10.1007/s13311-020-00839-0

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Zhou Y, Wang Y, Wang J, Anne Stetler R, Yang QW. Inflammation in intracerebral hemorrhage: from mechanisms to clinical translation. Prog Neurobiol. (2014) 115:25–44. doi: 10.1016/j.pneurobio.2013.11.003

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Lattanzi S, Brigo F, Trinka E, Cagnetti C, Di Napoli M, Silvestrini M. Neutrophil-to-lymphocyte ratio in acute cerebral hemorrhage: a system review. Transl Stroke Res. (2019) 10:137–45. doi: 10.1007/s12975-018-0649-4

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Oshiro EM, Walter KA, Piantadosi S, Witham TF, Tamargo RJ. A new subarachnoid hemorrhage grading system based on the Glasgow Coma Scale: a comparison with the Hunt and Hess and World Federation of Neurological Surgeons Scales in a clinical series. Neurosurgery. (1997) 41:140–7; discussion 147–8. doi: 10.1097/00006123-199707000-00029

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Jain S, Iverson LM. Glasgow Coma Scale. In: Babak A, et al. StatPearls. Treasure Island, FL: StatPearls Publishing Copyright, ©, 2020. StatPearls Publishing LLC. (2020).

Google Scholar

41. Zhao B, Cao Y, Tan X, Zhao Y, Wu J, Zhong M, et al. Complications and outcomes after early surgical treatment for poor-grade ruptured intracranial aneurysms: A multicenter retrospective cohort. Int J Surg. (2015) 23:57–61. doi: 10.1016/j.ijsu.2015.09.008

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Shen J, Huang K, Shen J, Zhu Y, Jiang H, Pan J, et al. Clinical efficacy between microsurgical clipping and endovascular coiling in the treatment of ruptured poor-grade anterior circulation aneurysms. World Neurosurg. (2019) 127:e321–9. doi: 10.1016/j.wneu.2019.02.248

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Olsen MH, Orre M, Leisner ACW, Rasmussen R, Bache S, Welling KL, et al. Delayed cerebral ischaemia in patients with aneurysmal subarachnoid haemorrhage: functional outcome and long-term mortality. Acta Anaesthesiol Scand. (2019) 63:1191–9. doi: 10.1111/aas.13412

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Budohoski KP, Guilfoyle M, Helmy A, Huuskonen T, Czosnyka M, Kirollos R, et al. The pathophysiology and treatment of delayed cerebral ischaemia following subarachnoid haemorrhage. J Neurol Neurosurg Psychiatry. (2014) 85:1343–53. doi: 10.1136/jnnp-2014-307711

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Engelhardt B, Vajkoczy P, Weller RO. The movers and shapers in immune privilege of the CNS. Nat Immunol. (2017) 18:123–31. doi: 10.1038/ni.3666

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Meyer C, Martin-Blondel G, Liblau RS. Endothelial cells and lymphatics at the interface between the immune and central nervous systems: implications for multiple sclerosis. Curr Opin Neurol. (2017) 30:222–30. doi: 10.1097/WCO.0000000000000454

PubMed Abstract | CrossRef Full Text | Google Scholar