Development and Validation of a Nomogram to Predict the Individual Future Stroke Risk for Adult Patients With Moyamoya Disease: A Multicenter Retrospective Cohort Study in China

Background: Studies exploring the predictive performance of major risk factors associated with future stroke events are insufficient, and a useful tool to predict individual risk is not available. Therefore, personalized advice for preventing future stroke in patients with moyamoya disease (MMD) cannot provide evidence-based recommendations. The aim of this study was to develop a novel nomogram with reliable validity to predict the individual risk of future stroke for adult MMD patients.

Methods: This study included 450 patients from seven medical centers between January 2013 and December 2018. Follow-ups were performed via clinical visits and/or telephone interviews from initial discharge to December 2019. The cohort was randomly assigned to a training set (2/3, n = 300) for nomogram development and a test set (1/3, n = 150) for external validation. The Kaplan-Meier analyses and receiver operating characteristic (ROC) curves were applied to assess the clinical benefits of this nomogram.

Results: Diabetes mellitus, a family history of MMD, a past history of stroke or transient ischemic attack, clinical manifestation, and treatment were identified as major risk factors via the least absolute shrinkage and selection operator (LASSO) method. A nomogram including these predictors was established via a multivariate Cox regression model, which displayed excellent discrimination [Harrell's concordance index (C-index), 0.85; 95% confidence interval (CI): 0.75–0.96] and calibration. In the external validation, the nomogram was found to have good discrimination (C-index, 0.81; 95% CI: 0.68–0.94) and calibration. In the subgroup analysis, this predictive nomogram also showed great performance in both ischemic-type (C-index, 0.90; 95% CI: 0.77–1.00) and hemorrhagic-type MMD (C-index, 0.72; 95% CI: 0.61–0.83). Furthermore, the nomogram was shown to have potential in clinical practice through Kaplan-Meier analyses and ROC curves.

Conclusions: We developed a novel nomogram incorporating several clinical characteristics with relatively good accuracy, which may have considerable potential for evaluating individual future stroke risk and providing useful management recommendations for adult patients with MMD in clinical practice.

Introduction

Moyamoya disease (MMD) is a rare chronic cerebrovascular disease characterized by progressive stenosis and/or occlusion of the distal internal carotid artery (ICA) and its major branches with the development of abnormal collateral circulation at the base of the brain (1). Although genotype-phenotype correlation analysis has revealed that ring finger protein 213 (RNF213) is a key susceptibility gene for MMD, the etiology and pathogenesis are still not fully understood (2, 3). The incidence of MMD is high in Asian countries, and presenting symptoms are different across regions (4). MMD is an important cause of stroke and is associated with a relatively high recurrence risk of cerebrovascular events and a poor prognosis (5, 6). Several independent risk factors, including an initial symptom of intraventricular hemorrhage, smoking, Asian ethnicity, a history of transient ischemic attack (TIA), and a reduced hemodynamic reserve, have been reported to significantly contribute to recurrent stroke in MMD (7–9). However, the small sample sizes of these trials may cause inconsistencies across studies that engender heterogeneity. Therefore, a single risk factor might be insufficient to predict the individual probability of cerebrovascular events in the future.

Recent evidence has demonstrated that surgical revascularization is more effective in preventing recurrent strokes and improving neurological functions mainly for hemorrhagic MMD patients than conservative treatment (10–13). However, the recurrent stroke risk has not markedly decreased in ischemic MMD patients undergoing revascularization (14). In addition, whether surgical revascularization should be applied to asymptomatic patients and the optimal time to perform surgery in conservatively treated patients remain controversial (6). Thus, credible prognostic information must be obtained for individual patients to facilitate lifestyle and/or personalized therapeutic decisions. A nomogram is a visual scoring model based on several risk factors used to generate a numerical probability of a particular clinical event for an individual patient in terms of disease prognosis (15).

In this study, we first identify the major risk factors associated with future stroke events from all predictors. Second, we develop and validate a nomogram incorporating these risk factors to calculate individualized predictions of the future stroke risk and provide important treatment recommendations for adult patients with MMD.

Materials and Methods

Study Design

A multicenter retrospective cohort study was conducted to identify the major risk factors and develop a prediction model to evaluate the individual risk of future stroke in adult patients with MMD. The research was approved by all participating medical center's ethics committee (2020[138]). Patient consent was not required because of the use of deidentified data. This study was carried out on the basis of the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement (16).

Patient Selection

Patients with a clinical diagnosis of MMD were identified from the national health care system using the diagnosis-specific codes (ICD-9 code, 437.5; ICD-10 code, I67.5) in seven participating academic medical centers between January 2013 and December 2018. The inclusion criteria were as follows: (1) a diagnosis of MMD based on the 2012 Tokyo criteria (6) [definitive MMD was defined as the bilateral stenosis and/or occlusion of the terminal portion of ICAs and/or the proximal portion of the anterior cerebral artery (ACA) and/or the middle cerebral artery (MCA) with the abnormal vascular network via cerebral digital subtraction angiography (DSA) and/or magnetic resonance angiography (MRA); probable MMD was defined as the unilateral involvement with the abnormal vascular network near the lesion via DSA while lacking predisposing factors for steno-occlusive changes]; (2) age 18 years or older; (3) no history of prior use of antiplatelet agents; (4) no history of prior neurosurgery; and (5) complete data for all predictors of interest. Patients were excluded from the study if they were diagnosed with moyamoya syndrome, had malignant tumors or were lost to follow-up.

Clinical Follow-Up

After initial discharge, long-term follow-ups were performed via clinical visits and/or telephone interviews until December 2019. The primary outcome was future stroke events defined as an acute focal infarction or hemorrhage of the brain, including sudden onset of a new focal neurological deficit and/or rapid worsening of an existing neurological deficit lasting 24 h or more with/without imaging evidence, while neurological dysfunctions lasting <24 h required imaging evidence by two independent neurologists. However, TIA, cognitive impairment, seizure, and perioperative stroke were excluded. The follow-up period was also obtained, which was defined as the time from initial discharge to the main outcome or December 2019.

Predictors

The baseline characteristics of the MMD patients were collected from electronic medical records by two independent neurologists who did not participate in the study design and data analysis; the characteristics included mainly age at the first symptom, age at diagnosis, sex, ethnicity, vascular risk factors (hypertension, diabetes mellitus, hyperlipidemia, smoking, and drinking), a past history of stroke or TIA, a family history of MMD, modified Rankin Scale (mRS) score, bilateral steno-occlusive change, Suzuki stage, intracranial aneurysm, posterior cerebral artery (PCA) involvement, collateral circulation (17) [absent, ICA-vertebral artery (VA) originated, external carotid artery (ECA) originated], clinical manifestation [other, TIA, lacunar infarction, cerebral infarction, intracerebral hemorrhage (ICH), subarachnoid hemorrhage (SAH)], and treatment choice (conservative treatment, antiplatelet therapy, surgical revascularization). Hypertension was defined as the systolic blood pressure (BP) ≥ 140 mmHg and/or diastolic BP ≥ 90 mmHg, or prior to taking antihypertensive agents. Diabetes mellitus was defined as the fasting glucose ≥ 7.0 mmol/L and/or 2 h-oral glucose tolerance test (OGTT) ≥ 11.1 mmol/L, or prior to using the antidiabetic agents. Hyperlipidemia was defined as total cholesterol ≥ 5.17 mmol/L, triglyceride ≥ 2.3 mmol/L, and/or low-density lipoprotein cholesterol ≥ 1.8 mmol/L, or prior to using statins. Heavy smoking was defined as smoking ≥ 1 cigarette per day and more than 1 year. Alcohol consumption was defined as drinking alcohol > 200 g per week regularly. With regard to the treatment choice, surgical revascularization was recommended to patients with non-emergency status, markedly neurological symptoms and no surgical contraindications, including direct bypass, indirect bypass, and combined bypass. Antiplatelet therapy was recommended to patients with ischemic MMD and who were not willing to undergo revascularization, including aspirin (100 mg/d), clopidogrel (75 mg/d), or combined agents for the first 3 weeks followed by aspirin daily. Conservatively treated patients also included other medications and surgery but not revascularization. Additionally, if a patient presented with an initial symptom of TIA, we did not add to the past history of TIA. If data were missing, then the patient was excluded.

Sample Size

An adequate sample size is required for nomograms to identify a significant effect estimate. If the primary outcome is binary, then the minimum frequencies of the two response levels must be more than 10 times the number of predictors according to Harrell's guidelines (18). In our study, a total of 19 predictors were considered. Thus, the sample size was at least 190 participants.

Development of the Nomogram

The participants were randomly assigned to the training set and the test set by R Project: 2/3 of the patients were used to develop the individualized risk prediction model, while the remaining 1/3 of the patients were used to validate the nomogram. First, the least absolute shrinkage and selection operator (LASSO) method was used to identify major risk factors associated with future strokes from all the predictors (19). The major risk factors were filtered via non-zero coefficients (20). Then, a prediction model incorporating the selected factors was established by a multivariate Cox regression analysis (21). These selected factors were reported as hazard ratios (HRs) with 95% confidence intervals (CIs) and P-values. The Cox model is given by the equation:

Individual risk (future stroke event at Y years) = baseline hazard (time = Y) ^*e^{β0*X0+β1*X1+β2*X2+…}.

Evaluation of the Nomogram

To evaluate the accuracy of the nomogram, discrimination and calibration were tested in this study. Discrimination was defined as the ability to distinguish patients with different outcomes. The Harrell's concordance index (C-index) was measured to evaluate this nomogram with bootstrapping validation (1,000 bootstrap resamples) (22). A higher value of the C-index indicated a more correct outcome of this nomogram, while a lower value suggested random chance. Then, the calibration was defined as the ability to reflect a difference between the predicted outcome and the actual outcome. Thus, a calibration plot was generated to present the predicted probabilities determined by the nomogram against the actual observed risk (23). An ideal prediction model would result in a 45-degree line showing perfect accuracy between predictive and observed outcomes. In addition, the absolute error of the nomogram was measured by the distance between an apparent plot and an ideal plot.

Validation of the Nomogram

The independent test set was used for external validation (24). The C-index was calculated to estimate the performance of the nomogram in predicting outcomes with 1,000 bootstrap resamples. The calibration of the predictive nomogram was assessed based on a calibration plot in the external validation (25).

Clinical Use of the Nomogram

Individual risk was calculated based on the prediction model in both the training set and the test set. Participants were classified into high-risk and low-risk groups according to the median value. The Kaplan-Meier analyses were applied to explore the stroke-free survival differences between both groups using the log-rank test (26). The receiver operating characteristic (ROC) curves were used to evaluate the 3-year performance of this nomogram (27).

Statistical Analysis

SPSS statistical software version 26 (IBM Corp) and R Project for Statistical Computing version 4.0.2 were used for statistical analyses. Statistical significance was defined by a 2-sided P-value below 0.05. Continuous variables were calculated as the median with the standard deviation. Categorical variables were reported as the number with proportions. The t-test and Mann-Whitney U test were used to analyze differences in continuous variables. The Pearson chi-square test was performed to explore differences in categorical variables. The LASSO method was applied to identify major risk factors. Multivariable Cox regression analysis was used to develop the predictive nomogram. The analysis began in January 2020 and ended in September 2020.

Results

Clinical Characteristics and Main Outcomes

This study was carried out as described in the flow chart (Figure 1). A total of 612 MMD patients were recruited at seven academic medical centers from January 2013 to December 2018. After applying the exclusion criteria, 450 patients (73.5%) who met the inclusion criteria and completed the clinical visits and/or telephone interviews through December 2019 were included in our final analysis (Supplementary Table 1). All patients were randomly assigned to either the training set (2/3 of the sample, n = 300; Supplementary Table 2) or the test set (1/3 of the sample, n = 150; Supplementary Table 3). Patient characteristics in both sets are provided in Table 1. No significant differences in clinical characteristics were found between the two datasets. Future stroke occurred in 46 patients (10.2%) during a follow-up of 34 ± 17 months in the present study. The detailed characteristics of these patients are summarized in Table 2. In addition, the demographic and clinical characteristics between patients with and without future stroke events are presented in Tables 3, 4. We found that a family history of MMD, a past history of stroke or TIA and treatment choice were remarkably different between both groups in the training and test sets, suggesting that these clinical features could be the potential risk factors for MMD with future strokes.

FIGURE 1

Figure 1. Flow chart presenting the process of patient inclusion and exclusion in this study.

TABLE 1

Table 1. The clinical characteristics of adult patients with moyamoya disease.

TABLE 2

Table 2. Summary of 46 patients with recurrent stroke.

TABLE 3

Table 3. Comparison of demographic and clinical characteristics between patients with and without future stroke events in the training group.

TABLE 4

Table 4. Comparison of demographic and clinical characteristics between patients with and without future stroke events in the test group.

The Major Risk Factors Associated With Future Stroke Events

Among texture features, a total of 5 major risk factors with non-zero coefficients were selected from the 19 potential predictors in the 300 adult patients with MMD in the training set using the LASSO regression model (1:4 ratio, Figure 2), including diabetes mellitus, a family history of MMD, a past history of stroke or TIA, clinical manifestation, and treatment choice.

FIGURE 2

Figure 2. Selection of major risk factors using the least absolute shrinkage and selection operator (LASSO) binary logistic regression model. (A) Tuning parameter (λ) selection in the LASSO model used 5-fold cross-validation via minimum criteria. The partial likelihood deviance (binomial deviance) curve was plotted vs. log(λ). Dotted vertical lines were drawn at the optimal values using the minimum criteria and the 1 standard error of the minimum criteria (the 1-SE criteria). (B) LASSO coefficient profiles of the 19 predictors. A coefficient profile plot was produced against the log(λ) sequence. A vertical line was drawn at the value selected using 5-fold cross-validation, where the optimal λ resulted in 6 non-zero coefficients.

Development of an Individualized Prediction Model

Diabetes mellitus, a family history of MMD, a past history of stroke or TIA, clinical manifestation and treatment were all identified as independent risk factors by multivariable Cox regression analysis (Table 5). Therefore, a predictive nomogram incorporating the above predictors was developed for calculating the individual probability of 3-year stroke-free survival and is presented in Figure 3.

TABLE 5

Table 5. The significant predictors of future stroke events in adult patients with moyamoya disease via a multivariate Cox regression analysis.

FIGURE 3

Figure 3. Development of a novel nomogram for predicting the individual risk of future stroke among adult patients with moyamoya disease (MMD) via a multivariate Cox regression analysis. The predictive nomogram was developed in the training set, with diabetes mellitus, a family history of MMD, a past history of stroke or TIA, clinical manifestation, and treatment incorporated.

Performance of the Nomogram in Predicting the Individual Future Stroke Risk

The C-index for the predictive nomogram was 0.85 (95% CI: 0.75–0.96) by bootstrapping validation in the training set, indicating good discrimination of this nomogram. The calibration plot of this prediction model revealed good accordance between actual observations and nomogram predictions in the training set (Figure 4A). In this nomogram, the apparent performance exhibited high predictive power.

FIGURE 4

Figure 4. Calibration curves of the predictive nomogram. (A) Calibration curve of the predictive nomogram for 3-year stroke-free survival in the training set. (B) Calibration curve of the predictive nomogram for 3-year stroke-free survival in the test set. (C) Calibration curve of the predictive nomogram for 3-year stroke-free survival in the ischemic-type group. (D) Calibration curve of the predictive nomogram for 3-year stroke-free survival in the hemorrhagic-type group. Calibration curves depict the calibration of each model in terms of the agreement between the predicted risks of future stroke and observed outcomes of future stroke. The x-axis represents the predicted future stroke risk. The y-axis represents the actual observed rate of future stroke. The diagonal dotted line represents a perfect prediction by an ideal model. The solid line represents the performance of the nomogram, where a closer fit to the diagonal dotted line represents a better prediction.

Validation of the Nomogram in Predicting the Individual Future Stroke Risk

The C-index of the nomogram for predicting the individual future stroke risk was 0.81 (95% CI: 0.68–0.94) in the test set for external validation. Good calibration was also observed in the test set (Figure 4B).

Clinical Use of the Nomogram

On the basis of the prediction model, patients were classified into high-risk and low-risk groups via a cutoff of mean value. Patients in the high-risk group suffered more future stroke risks than those in the low-risk group (Training: 18.0 vs. 1.3%; Test: 22.2 vs. 3.4%). The Kaplan-Meier plots also demonstrated that patients in the high-risk group were associated with poor stroke-free survival in Figures 5A,B. The 3-year area under the curve (AUC) values were 0.79 in the training set (Figure 5E) and 0.80 in the test set (Figure 5F). In general, this nomogram indicated a good predictive accuracy of the individual future stroke risk in clinical practice.

FIGURE 5

Figure 5. Evaluation of the clinical performance of the nomogram. (A) The Kaplan-Meier plot of the predictive nomogram in the training set. (B) The Kaplan-Meier plot of the predictive nomogram in the test set. (C) The Kaplan-Meier plot of the predictive nomogram in the ischemic-type group. (D) The Kaplan-Meier plot of the predictive nomogram in the hemorrhagic-type MMD. (E) ROC curves for predictions of 3-year stroke-free survival in the training set. (F) ROC curves for predictions of 3-year stroke-free survival in the test set. (G) ROC curves for predictions of 3-year stroke-free survival in the ischemic-type group. (H) ROC curves for predictions of 3-year stroke-free survival in the hemorrhagic-type MMD.

Subgroup Analysis

In the subgroup analysis, we independently evaluated the performance of this nomogram in the 187 ischemic-type and 216 hemorrhagic-type MMD patients. Future stroke occurred in 18 ischemic-type patients (9.6%) during a follow-up of 33 ± 18 months and in 27 hemorrhagic-type patients (12.5%) during a follow-up of 35 ± 16 months (Supplementary Tables 4, 5). This predictive nomogram showed great performance in both ischemic-type (C-index, 0.90; 95% CI: 0.77–1.00) and hemorrhagic-type MMD (C-index, 0.72; 95% CI: 0.61–0.83) with excellent calibration (Figures 4C,D). Additionally, the individual risk score was calculated according to this nomogram, which was used to divide these patients into high-risk and low-risk groups on the basis of the mean value. The survival analyses showed that the stroke-free survival was significantly higher in the low-risk group in both ischemic-type and hemorrhagic-type groups (Figures 5C,D). The ROC curves also demonstrated good 3-year AUC in the ischemic-type group (AUC = 0.84) and hemorrhagic-type group (AUC = 0.74) (Figures 5G,H). In short, this nomogram could be independently used in different MMD subgroups with a great predictive ability.

Discussion

In this study, the prediction model incorporating major risk factors performed well in predicting individual future stroke risk in adult MMD patients with relatively good accuracy. Additionally, the model may act as an available scoring system to distinguish the risk level and help doctors to provide treatment recommendations. Although many observational studies have revealed the clinical features and therapeutic methods for MMD, few have focused on the risk factors involved in prognosis. These findings from previous studies were not unique due to small sample sizes. This study was designed with a total of 450 patients from seven centers to acquire reliable results.

Similar to previous studies, our study has demonstrated that a pre-existing history of stroke or TIA might be an independent risk factor associated with future stroke events. Except for chronic steno-occlusive changes of the circle of Willis, Lee et al. (28) has revealed autoregulatory dysfunctions of BP in MMD patients, which could cause a fast or persistent cerebral ischemia symptom. However, the proportion of patients with a pre-existing history of stroke or TIA in our study was distinctly lower than the proportion in the study by Hervé et al. (8). Furthermore, patients with TIA could not be recorded because such mild symptoms that manifested with a rapidly recovery were easily ignored by patients; therefore, it is likely that this could explain the difference. In addition, the clinical manifestation was identified as a risk factor in this study. Hemorrhagic symptoms were the most common disease type in adult MMD patients with a slight advantage over ischemic symptoms. The hemorrhagic type of ICH and SAH constitute an important factor associated with a relatively higher risk of future strokes in both our study and the study by Kim et al. study (7). ICH originated from the abnormal moyamoya vessels due to the developmental functional disorders of vascular endothelial cells (29). Hemorrhage sites from the posterior circulation have a higher recurrence risk than from the anterior circulation (30), and the intracranial aneurysm (IA) could increase the hemorrhagic risk for MMD patients (31). Although they were not identified as risk factors of future strokes, our study found that hemorrhagic-type MMD patients with IA showed relatively higher recurrent stroke risks than those without IA (15.2 vs. 12.0%). Additionally, the ischemic type of TIA and cerebral infarction directly reflects the severity of stenosis and/or occlusion in the intracranial arteries, while patients with an initial symptom of lacunar infarction show a very low recurrent stroke risk, which is similar to the report from Zhao et al. (32), suggesting that lacunar infarction may act as a biomarker associated with a favorable long-term prognosis of low-risk recurrence. Moreover, our study has also found that surgical revascularization was an effective strategy for adult MMD in preventing future strokes (conservative vs. antiplatelet vs. surgical = 13.5 vs. 8.1 vs. 4.1%; Log-rank test: Chi-square = 5.404, p = 0.02; Supplementary Figures 1A,B). Even though the use of antiplatelet agents in the treatment of MMD remains controversial, it was interesting that antiplatelet therapy reduced the future stroke risk compared with surgical and conservative treatments for ischemic-type patients in our study (conservative vs. antiplatelet vs. surgical = 3.8 vs. 8.5 vs. 14.5%; Log-rank test: Chi-square = 4.717, p = 0.03; Supplementary Figures 1C,D). Recent evidence has demonstrated that luminal thrombosis is a pathological feature of MMD and contributes to steno-occlusive changes in the distal ICA (1). In addition, antiplatelet agents have been widely applied for preventive management in acute ischemic stroke and TIA (33, 34). Thus, more clinical studies should be performed to investigate the efficacy and safety of antiplatelet agents in the future.

In contrast to previous studies, our study observed that a family history of MMD and diabetes mellitus were significantly associated with future strokes. The proportion of patients with a family history of MMD was ~ 5.6%, which is consistent with the report from Duan et al. (5), indicating that a genetic factor might play a role in the pathogenesis of MMD. Although no obvious differences were apparent between MMD patients and the general population, diabetes mellitus was significantly correlated with severe cerebrovascular disease and contributed to cerebral neovascularization. Ren et al. (35) report that adult MMD patients with diabetes mellitus acquire better cerebral perfusion after revascularization. However, one different result from an observational study with a large sample size demonstrates that diabetes mellitus acts as a risk factor for recurrent strokes in young adults with MMD after revascularization (36). Therefore, the effect of diabetes mellitus in patients with MMD remains unclear. Our study supports diabetes mellitus as a risk factor for recurrent stroke, regardless of the treatment choice. Although an invalidated hypothesis suggests that diabetes mellitus is associated with elevated growth factors and cytokines causing the formation of collateral circulation to improve the blood supply, the major problem for patients with MMD is the chronic steno-occlusive change of ICAs causing the cerebral hypoperfusion and that abnormal angiogenesis could deteriorate the cerebral ischemia. Thus, additional randomized case-control studies are necessary to evaluate the predictive effectiveness of these factors in the future.

Several clinical benefits are associated with the application of the predictive nomogram. The nomogram can be easily used to calculate the individual future stroke risk and provide important treatment recommendations in clinical practice. For patients with a low risk of future stroke, blood sugar monitoring and a long-term medical follow-up should be recommended. For high-risk patients, personalized therapy is required to provide more appropriate treatment to reduce the risk level and improve the prognostic outcome. In addition to an outpatient follow-up and blood sugar control, therapeutic changes are necessary for patients. With regard to hemorrhagic-type MMD, surgical revascularization is employed to prevent recurrent stroke and improve neurological dysfunctions (10–13). For ischemic-type MMD, antiplatelet therapy is inferior to surgical revascularization while superior to conservative management for changing the blood supply and preventing future stroke events (37). Revascularization surgery is capable of reducing the recurrence of future stroke for patient with hemorrhagic MMD, but it has not shown an advantage over conservative treatment for patients with ischemic MMD (14, 38). Many patients are not willing to undergo surgery because of their mild symptoms and/or high operative risks in clinical practice. Although antiplatelet therapy remains controversial, our study shows a positive result that proves its capacity, especially for patients with the initial symptom of cerebral infarction. Additionally, this nomogram provides prognostic information for doctors to identify high-risk patients receiving antiplatelet agents and prompts the selection of revascularization as a replacement treatment. With regard to asymptomatic MMD, revascularization and antiplatelet therapy are also controversial, and clinical studies demonstrating the effectiveness of such treatments are not abundant. Thus, doctors can encourage these treatments as supplementary therapy to prevent future strokes.

The strengths of the current study include a relatively larger number of enrolled patients, the long-term follow-up, and the broad scope of the clinical data collection from seven high-level hospitals across China. However, several limitations exist in this study. The genomic characteristics were not included in our analysis. Some genome-wide and locus-specific association studies identified that RNF213 polymorphism was an important genetic risk factor for MMD in East Asian population (39) which may influence the clinical phenotypes of MMD (40). Several meta-analysis studies shown that RNF213 p.4810K significantly and markedly increased MMD risk (Odds ratio = 60–100) (41–43). Even though the precise mechanism of the RNF213 variant in MMD pathogenesis is not clear (44), genetic epidemiological studies have rigorously identified RNF213 p.R4810K as a risk factor in MMD patients. In addition, this variant was reported to increase the risk of ischemic stroke due to large-artery atherosclerosis (45). Certain previous studies have reported that ~1% of the general population were unaffected genetic carriers in the East Asian population (4), but the gene was also associated with other diseases (46, 47), suggesting that RNF213 might not be a specific gene for MMD. However, the incidence rate of RNF213 polymorphism in MMD was proven relatively higher (0.41 per 100,000, 95% CI: 0.28–0.54) in the Chinese population and the effect of this variant was very important for MMD patients (4). Thus, an insufficient survey of RNF213 polymorphism is a limitation in this study and RNF213 variant should be investigated in the future. Furthermore, some potential risk factors, such as vascular endothelial growth factor (VEGF) and matrix metalloprotein (MMP) status, which are not routinely evaluated, were not included. In contrast to a previous study, computed tomography perfusion (CTP) imaging and positron emission tomography (PET) scans for hemodynamic reserve and cerebral perfusion are not conducted in most Chinese patients due to their high cost and medical insurance limitations. Moreover, the proportion of patients who underwent surgical revascularization was lower than previous studies; in addition, the rate was substantially lower in patients from western China than in those from southern coastal China. In addition to the presence of mild symptoms and fears regarding surgery, education level and economic income might also result in this difference. Based on this study, the individual risk score could be calculated using this prediction model as follows: risk score = diabetes mellitus × 1.12566516 + a family history of MMD × 2.04512987 + a past history of TIA or stroke × 1.50137252 + clinical manifestation × 0.5129745 + treatment × (−0.6259136). Since the individual risk score was a skewed distribution, the median value (M) and interquartile range (IQR) were used to describe the normal range of the risk score in this study (M = 1.68, IQR = 0.36–1.85). Importantly, since our prediction model was entirely based on Chinese patients, the generalizability of this risk score in the areas outside of China is limited. However, these clinical features could be easily recorded in clinical practice, so our study may provide scientists outside of China with a new and viable approach in further studies. It is possible that the prediction models of other ethnic groups differ from those of the Chinese population. Ultimately, some aspects are not completely known, such as environmental factors and other conditions.

In conclusion, our study presents a novel nomogram incorporating several major risk factors with relatively good accuracy. The nomogram can be conveniently used to help neurologists and neurosurgeons evaluate the individual risk of a future stroke event and provide useful treatment recommendations for adult MMD patients in clinical practice.

Data Availability Statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author/s.

Ethics Statement

The studies involving human participants were reviewed and approved by the independent ethics committee of the First Affiliated Hospital of Sun Yat-sen University and each participating medical center's ethics committee (number: [2020]138). Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

Author Contributions

FY, TW, and HY designed the study, drafted the manuscript, and contributed to the discussion. JL analyzed the data. HL, TG, XZ, TY, LJ, and XW collected the data. WS and QL designed the study, reviewed the manuscript, and contributed to the discussion. All authors contributed to the article and approved the submitted version.

Funding

This study was supported by the National Natural Science Foundation of China (Nos. 81671132, 81471180, and 82071286).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors thank Weidong Li, Ph.D. (Department of Statistics, School of Public Health, Sun Yat-sen University), for statistical assistance.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fneur.2021.669025/full#supplementary-material

Supplementary Figure 1. The survival analysis of future strokes in adult patients with moyamoya disease (MMD). (A) The cumulative stroke-free survival in all the MMD patients (Log-rank test: Chi-square = 5.40, p = 0.02). (B) The cumulative risk in all the MMD patients. (C) The cumulative stroke-free survival in the ischemic-type MMD patients (Log-rank test: Chi-square = 4.72, p = 0.03). (D) The cumulative risk in the ischemic-type MMD patients.

Supplementary Table 1. The original data of 450 adult patients with moyamoya disease.

Supplementary Table 2. The training set of 300 adult patients with moyamoya disease.

Supplementary Table 3. The test set of 150 adult patients with moyamoya disease.

Supplementary Table 4. The original data of 187 adult patients with ischemic-type moyamoya disease.

Supplementary Table 5. The original data of 216 adult patients with hemorrhagic-type moyamoya disease.

References

1. Scott RM, Smith ER. Moyamoya disease and moyamoya syndrome. N Engl J Med. (2009) 360:1226–37. doi: 10.1056/NEJMra0804622

CrossRef Full Text | Google Scholar

2. Liu W, Morito D, Takashima S, Mineharu Y, Kobayashi H, Hitomi T, et al. Identification of RNF213 as a susceptibility gene for moyamoya disease and its possible role in vascular development. PLoS ONE. (2011) 6:e22542. doi: 10.1371/journal.pone.0022542

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Bang OY, Fujimura M, Kim SK. The pathophysiology of moyamoya disease: an update. J Stroke. (2016) 18:12–20. doi: 10.5853/jos.2015.01760

CrossRef Full Text | Google Scholar

4. Kleinloog R, Regli L, Rinkel GJ, Klijn CJ. Regional differences in incidence and patient characteristics of moyamoya disease: a systematic review. J Neurol Neurosurg Psychiatry. (2012) 83:531–6. doi: 10.1136/jnnp-2011-301387

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Duan L, Bao XY, Yang WZ, Shi WC, Li DS, Zhang ZS, et al. Moyamoya disease in China: its clinical features and outcomes. Stroke. (2012) 43:56–60. doi: 10.1161/STROKEAHA.111.621300

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Research Committee on the Pathology and Treatment of Spontaneous Occlusion of the Circle of Willis and Health Labour Sciences Research Grant for Research on Measures for Infractable Diseases. Guidelines for diagnosis and treatment of moyamoya disease (spontaneous occlusion of the circle of Willis). Neurol Med Chir. (2012) 52:245–66. doi: 10.2176/nmc.52.245

CrossRef Full Text | Google Scholar

7. Kim KM, Kim JE, Cho WS, Kang HS, Son YJ, Han MH, et al. Natural history and risk factor of recurrent hemorrhage in hemorrhagic adult moyamoya disease. Neurosurgery. (2017) 81:289–96. doi: 10.1093/neuros/nyw179

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Hervé D, Ibos-Augé N, Calvière L, Rogan C, Labeyrie MA, Guichard JP, et al. Predictors of clinical or cerebral lesion progression in adult moyamoya angiopathy. Neurology. (2019) 93:e388–97. doi: 10.1212/WNL.0000000000007819

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Kang S, Liu X, Zhang D, Wang R, Zhang Y, Zhang Q, et al. Natural course of moyamoya disease in patients with prior hemorrhagic stroke. Stroke. (2019) 50:1060–6. doi: 10.1161/STROKEAHA.118.022771

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Liu X, Zhang D, Shuo W, Zhao Y, Wang R, Zhao J. Long term outcome after conservative and surgical treatment of haemorrhagic moyamoya disease. J Neurol Neurosurg Psychiatry. (2013) 84:258–65. doi: 10.1136/jnnp-2012-302236

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Cho WS, Kim JE, Kim CH, Ban SP, Kang HS, Son YJ, et al. Long-term outcomes after combined revascularization surgery in adult moyamoya disease. Stroke. (2014) 45:3025–31. doi: 10.1161/STROKEAHA.114.005624

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Acker G, Fekonja L, Vajkoczy P. Surgical management of moyamoya disease. Stroke. (2018) 49:476–82. doi: 10.1161/STROKEAHA.117.018563

CrossRef Full Text | Google Scholar

13. Ando T, Shimada Y, Fujiwara S, Yoshida K, Kobayashi M, Ogasawara K. Revascularisation surgery improves cognition in adult patients with moyamoya disease. J Neurol Neurosurg Psychiatry. (2020) 91:332–4. doi: 10.1136/jnnp-2019-321069

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Liu XJ, Zhang D, Wang S, Zhao YL, Teo M, Wang R, et al. Clinical features and long-term outcomes of moyamoya disease: a single-center experience with 528 cases in China. J Neurosurg. (2015) 122:392–9. doi: 10.3171/2014.10.JNS132369

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Iasonos A, Schrag D, Raj GV, Panageas KS. How to build and interpret a nomogram for cancer prognosis. J Clin Oncol. (2008) 26:1364–70. doi: 10.1200/JCO.2007.12.9791

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ. (2015) 350:g7594. doi: 10.1136/bmj.g7594

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Zhao M, Zhang D, Wang S, Zhang Y, Deng X, Zhao J. The collateral circulation in moyamoya disease: a single-center experience in 140 pediatric patients. Pediatr Neurol. (2017) 77:78–83. doi: 10.1016/j.pediatrneurol.2017.08.016

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Kuonen D. Book review: regression modeling strategies: with applications to linear models, logistic regression, and survival analysis. Stat Methods Med Res. (2004) 13:415–6. doi: 10.1177/096228020401300512

CrossRef Full Text | Google Scholar

19. Sauerbrei W, Royston P, Binder H. Selection of important variables and determination of functional form for continuous predictors in multivariable model building. Stat Med. (2007) 26:5512–28. doi: 10.1002/sim.3148

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Kidd AC, McGettrick M, Tsim S, Halligan DL, Bylesjo M, Blyth KG. Survival prediction in mesothelioma using a scalable Lasso regression model: instructions for use and initial performance using clinical predictors. BMJ Open Respir Res. (2018) 5:e000240. doi: 10.1136/bmjresp-2017-000240

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Wang Y, Li J, Xia Y, Gong R, Wang K, Yan Z, et al. Prognostic nomogram for intrahepatic cholangiocarcinoma after partial hepatectomy. J Clin Oncol. (2013) 31:1188–95. doi: 10.1200/JCO.2012.41.5984

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Pencina MJ, D'Agostino RB. Overall C as a measure of discrimination in survival analysis: model specific population value and confidence interval estimation. Stat Med. (2004) 23:2109–23. doi: 10.1002/sim.1802

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Alba AC, Agoritsas T, Walsh M, Hanna S, Iorio A, Devereaux PJ, et al. Discrimination and calibration of clinical prediction models: users' guides to the medical literature. JAMA. (2017) 318:1377–84. doi: 10.1001/jama.2017.12126

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Hyman DM, Eaton AA, Gounder MM, Smith GL, Pamer EG, Hensley ML, et al. Nomogram to predict cycle-one serious drug-related toxicity in phase I oncology trials. J Clin Oncol. (2014) 32:519–26. doi: 10.1200/JCO.2013.49.8808

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Huang YQ, Liang CH, He L, Tian J, Liang CS, Chen X, et al. Development and validation of a radiomics nomogram for preoperative prediction of lymph node metastasis in colorectal cancer. J Clin Oncol. (2016) 34:2157–64. doi: 10.1200/JCO.2015.65.9128

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Kaplan EL, Meier P. Nonparametric estimation from incomplete observations. J Am Stat Assoc. (1958) 53:457–62. doi: 10.1080/01621459.1958.10501452

CrossRef Full Text | Google Scholar

27. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. (1982) 143:29–36. doi: 10.1148/radiology.143.1.7063747

CrossRef Full Text | Google Scholar

28. Lee JK, Williams M, Reyes M, Ahn ES. Cerebrovascular blood pressure autoregulation monitoring and postoperative transient ischemic attack in pediatric moyamoya vasculopathy. Paediatr Anaesthesia. (2018) 28:94–102. doi: 10.1111/pan.13293

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Choi JW, Son SM, Mook-Jung I, Moon YJ, Lee JY, Wang KC, et al. Mitochondrial abnormalities related to the dysfunction of circulating endothelial colony-forming cells in moyamoya disease. J. Neurosurg. (2018) 129:1151–9. doi: 10.3171/2017.5.JNS17147

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Takahashi JC, Funaki T, Houkin K, Inoue T, Ogasawara K, Nakagawara J, et al. Significance of the hemorrhagic site for recurrent bleeding: prespecified analysis in the japan adult moyamoya trial. Stroke. (2016) 47:37–43. doi: 10.1161/STROKEAHA.115.010819

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Yang H, Zhang L, Wang M, Wang J, Chen L, Lu H. Clinical features of and risk factors for intracranial aneurysms associated with moyamoya disease. Int J Stroke. (2020). doi: 10.1177/1747493020967224. [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Zhao M, Deng X, Wang S, Zhang D, Zhang Y, Zhao J. Lacunar infarction in adult patients with moyamoya disease. Clin Neurol Neurosurg. (2018) 164:81–6. doi: 10.1016/j.clineuro.2017.10.040

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Kernan WN, Ovbiagele B, Black HR, Bravata DM, Chimowitz MI, Ezekowitz M, et al. Guidelines for the prevention of stroke in patients with stroke and transient ischemic attack: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke. (2014) 45:2160–236. doi: 10.1161/STR.0000000000000024

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Powers WJ, Rabinstein AA, Ackerson T, Adeoye OM, Bambakidis NC, Becker K, et al. 2018 Guidelines for the early management of patients with acute ischemic stroke: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke. (2018) 49:e46–110. doi: 10.1016/j.jvs.2018.04.007

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Ren B, Zhang ZS, Liu WW, Bao XY, Li DS, Han C, et al. Surgical outcomes following encephaloduroarteriosynangiosis in adult moyamoya disease associated with Type 2 diabetes. J Neurosurg. (2016) 125:308–14. doi: 10.3171/2015.7.JNS15218

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Zhao M, Deng X, Gao F, Zhang D, Wang S, Zhang Y, et al. Ischemic stroke in young adults with moyamoya disease: prognostic factors for stroke recurrence and functional outcome after revascularization. World Neurosurg. (2017) 103:161–7. doi: 10.1016/j.wneu.2017.03.146

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Ye F, Li J, Wang T, Lan K, Li H, Yin H, et al. Efficacy and safety of antiplatelet agents for adult patients with ischemic moyamoya disease. Front Neurol. (2021) 11:608000. doi: 10.3389/fneur.2020.608000

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Wouters A, Smets I, Van den Noortgate W, Steinberg GK, Lemmens R. Cerebrovascular events after surgery versus conservative therapy for moyamoya disease: a meta-analysis. Acta Neurol Belg. (2019) 119:305–13. doi: 10.1007/s13760-019-01165-9

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Koizumi A, Kobayashi H, Hitomi T, Harada KH, Habu T, Youssefian S. A new horizon of moyamoya disease and associated health risks explored through RNF213. Environ Health Prev Med. (2016) 21:55–70. doi: 10.1007/s12199-015-0498-7

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Wang Y, Yang L, Wang X, Zeng F, Zhang K, Zhang Q, et al. Meta-analysis of genotype and phenotype studies to confirm the predictive role of the RNF213 p.R4810K variant for moyamoya disease. Eur J Neurol. (2021) 28:823–36. doi: 10.1111/ene.14635

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Liao X, Deng J, Dai W, Zhang T, Yan J. Rare variants of RNF213 and moyamoya/non-moyamoya intracranial artery stenosis/occlusion disease risk: a meta-analysis and systematic review. Environ Health Prev Med. (2017) 22:75. doi: 10.1186/s12199-017-0680-1

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Wang Y, Mambiya M, Li Q, Yang L, Jia H, Han Y, et al. RNF213 p.R4810K polymorphism and the risk of moyamoya disease, intracranial major artery stenosis/occlusion, and quasi-moyamoya disease: a meta-analysis. J Stroke Cerebrovascu Dis. (2018) 27:2259–70. doi: 10.1016/j.jstrokecerebrovasdis.2018.04.013

CrossRef Full Text | Google Scholar

43. Wang X, Wang Y, Nie F, Li Q, Zhang K, Liu M, et al. Association of genetic variants with moyamoya disease in 13 000 individuals: a meta-analysis. Stroke. (2020) 51:1647–55. doi: 10.1161/STROKEAHA.120.029527

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Fujimura M, Sonobe S, Nishijima Y, Niizuma K, Sakata H, Kure S, et al. Genetics and biomarkers of moyamoya disease: significance of RNF213 as a susceptibility gene. J Stroke. (2014) 16:65–72. doi: 10.5853/jos.2014.16.2.65

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Okazaki S, Morimoto T, Kamatani Y, Kamimura T, Kobayashi H, Harada K, et al. Moyamoya disease susceptibility variant RNF213 p.R4810K increases the risk of ischemic stroke attributable to large-artery atherosclerosis. Circulation. (2019) 139:295–8. doi: 10.1161/CIRCULATIONAHA.118.038439

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Kim HJ, Choi EH, Chung JW, Kim JH, Kim YS, Seo WK, et al. Role of the RNF213 variant in vascular outcomes in patients with intracranial atherosclerosis. J Am Heart Assoc. (2021) 10:e017660. doi: 10.1161/JAHA.120.017660

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Zhou S, Ambalavanan A, Rochefort D, Xie P, Bourassa CV, Hince P, et al. RNF213 is associated with intracranial aneurysms in the French-Canadian population. Am J Hum Genet. (2016) 99:1072–85. doi: 10.1016/j.ajhg.2016.09.001

PubMed Abstract | CrossRef Full Text | Google Scholar