Classifying Ruptured Middle Cerebral Artery Aneurysms With a Machine Learning Based, Radiomics-Morphological Model: A Multicentral Study

Objective: Radiomics and morphological features were associated with aneurysms rupture. However, the multicentral study of their predictive power for specific-located aneurysms rupture is rare. We aimed to determine robust radiomics features related to middle cerebral artery (MCA) aneurysms rupture and evaluate the additional value of combining morphological and radiomics features in the classification of ruptured MCA aneurysms.

Methods: A total of 632 patients with 668 MCA aneurysms (423 ruptured aneurysms) from five hospitals were included. Radiomics and morphological features of aneurysms were extracted on computed tomography angiography images. The model was developed using a training dataset (407 patients) and validated with the internal (152 patients) and external validation (73 patients) datasets. The support vector machine method was applied for model construction. Optimal radiomics, morphological, and clinical features were used to develop the radiomics model (R-model), morphological model (M-model), radiomics-morphological model (RM-model), clinical-morphological model (CM-model), and clinical-radiomics-morphological model (CRM-model), respectively. A comprehensive nomogram integrating clinical, morphological, and radiomics predictors was generated.

Results: We found seven radiomics features and four morphological predictors of MCA aneurysms rupture. The R-model obtained an area under the receiver operating curve (AUC) of 0.822 (95% CI, 0.776, 0.867), 0.817 (95% CI, 0.744, 0.890), and 0.691 (95% CI, 0.567, 0.816) in the training, temporal validation, and external validation datasets, respectively. The RM-model showed an AUC of 0.848 (95% CI, 0.810, 0.885), 0.865 (95% CI, 0.807, 0.924), and 0.721 (95% CI, 0.601, 0.841) in the three datasets. The CRM-model obtained an AUC of 0.856 (95% CI, 0.820, 0.892), 0.882 (95% CI, 0.828, 0.936), and 0.738 (95% CI, 0.618, 0.857) in the three datasets. The CRM-model and RM-model outperformed the CM-model and M-model in the internal datasets (p < 0.05), respectively. But these differences were not statistically significant in the external dataset. Decision curve analysis indicated that the CRM-model obtained the highest net benefit for most of the threshold probabilities.

Conclusion: Robust radiomics features were determined related to MCA aneurysm rupture. The RM-model exhibited good ability in classifying ruptured MCA aneurysms. Integrating radiomics features into conventional models might provide additional value in ruptured MCA aneurysms classification.

Introduction

Middle cerebral artery (MCA) aneurysm is the most common subtype of unruptured aneurysms (Huttunen et al., 2010; Can et al., 2015). With the improvement of imaging techniques, unruptured aneurysms have become more frequently detected (Greving et al., 2014). But therapeutic decision-making for them is controversial. On the one hand, many unruptured aneurysms stay asymptomatic and never rupture (Korja et al., 2014). The prophylactic treatment such as current endovascular and microsurgical interventions carries the risk of procedure-related complications (Naggara et al., 2010; Zhu et al., 2020). On the other hand, once the aneurysm ruptures, the outcome is catastrophic (Vlak et al., 2011). Therefore, it is vital to screen out rupture-prone aneurysms.

Previous studies have identified that morphological features were associated with aneurysms rupture (Lindgren et al., 2016; Zhu et al., 2020). Researchers have constructed various computational methods using morphological features to evaluate the aneurysms-rupture risk (Zhang et al., 2019; Tanioka et al., 2020; Zhu et al., 2020). However, those morphological features are measured on two-dimensional images and might be affected by different readers or projections. It could impair the comparability of results.

Radiomics is an emerging technology that extracts high-throughput data from medical images (Zhou et al., 2018; Hua et al., 2020; Tomaszewski and Gillies, 2021). Recently, radiomics is frequently used in cerebrovascular disease researches (Chen et al., 2021; Zhu D. et al., 2021; Zhu D. Q. et al., 2021). Several researchers have scoped to the whole-brain aneurysms and proved that radiomics features were related to aneurysms rupture status (Liu et al., 2019; Ou et al., 2021). Regretfully, they did not analyze the features’ robustness, which can be easily affected by a slight change in image-scanning protocols or regions of interest (ROIs) segmentation (Mackin et al., 2015; Choe et al., 2019). Moreover, the predictive ability of radiomics in those studies was not validated by any external validation dataset, which leads to the uncertainness of their results’ generalizability (Collins et al., 2015; Lambin et al., 2017).

To the best of our knowledge, few studies have predicted the rupture of the location-specific aneurysm with robust radiomics features. In this study, we included a large sample of 668 MCA aneurysms. We aimed to (1) determine whether there are robust radiomics features that can classify ruptured MCA aneurysms; and (2) evaluate the additional value of combining morphological and radiomics features in classifying ruptured MCA aneurysm.

Materials and Methods

Our study was approved by the Medical Ethics Committee of our hospital.

Study Population and Clinical Data

We performed a retrospective and multicentral study using the data from five hospitals (hospitals A, B, C, D, and E). MCA aneurysms with available computed tomography angiography (CTA) data were included. Exclusion criteria were as follows: fusiform MCA aneurysms, aneurysms combined with vascular diseases (such as Moyamoya disease and arteriovenous malformations), aneurysms with a size <3 mm, aneurysms with poor-quality images and patients underwent surgery or interventional therapy before CTA examination (see Supplementary Methods, Supplementary Digital Content 1, which illustrates details about CTA image scanning).

Patients with MCA aneurysms seen in hospital A from January 2009 to December 2019 were allocated to the training and the internal validation datasets. The training dataset encompassed the patients from the earlier period (2009–2017). The patients from the more recent period (2018–2019) were attributed to the temporal validation dataset (internal validation dataset) (Collins et al., 2015; Al-Shahi Salman et al., 2018). For external validation, MCA aneurysms cases in four hospitals, including hospital B (from January 2018 to December 2020), hospital C (from January 2018 to December 2020), hospital D (from January 2017 to October 2019), and hospital E (from September 2019 to March 2020), were merged to one external validation dataset (Lambin et al., 2017).

Clinical data such as age, sex, history of hypertension (a diagnosis of hypertension previously made by another physician or use of antihypertensive drugs), cigarette smoking (previous smoker or current smoker), and aneurysm side were collected. Rupture status of aneurysms was evaluated using the following criterion: (1) for patients with subarachnoid hemorrhage (SAH), aneurysms adjacent to the cisternal clots were judged ruptured, and those aneurysms not adjacent to the cisternal clots were judged on digital subtraction angiography (DSA). (2) Asymptomatic patients without SAH were identified to be unruptured (Shi et al., 2021).

Morphological Predictors Discovery

Morphological features such as aneurysm location (divided into M1, the proximal segment of the middle cerebral artery; Mbif, main middle cerebral artery bifurcation; Mdist, distal middle cerebral artery), aneurysm size, vessel size, aneurysm height, perpendicular height, aspect ratio (AR), size ratio (SR), aneurysm angle, flow angle, vessel angle, daughter dome, and irregular shape were measured as described in previous studies (Can et al., 2015; Chen et al., 2020; Zhu et al., 2020; Figure 1C; see Supplementary Methods, Supplementary Digital Content 2, which illustrates detailed definitions of morphological features).

FIGURE 1

Figure 1. A flowchart of radiomics analysis and radiomics-clinical model construction. (A) The process of regions of interest (ROIs) segmentation; (B) the process of optimal radiomics features detection; (C) the process of morphological predictors discovery; and (D) the process of machine learning models development and validation. GLCM, gray level co-occurrence matrix; GLDM, gray level dependence matrix; GLRLM, grey level run-length matrix; GLSZM, gray level size zone matrix; NGTDM, neighboring gray tone difference matrix; SVM, support vector machine.

We implemented univariate analysis to find morphological factors that were associated with MCA aneurysm rupture. After that, the multivariable logistic regression was performed to identify independent morphological predictors of MCA aneurysm rupture.

Optimal Radiomics Signature Detection

The workflow process of radiomics analysis is shown in Figures 1A,B. ROIs of aneurysms were manually segmented by a neuroradiologist on each slice of CTA images (Figure 1A). Then, 50 aneurysms were randomly selected to be re-segmented by another neuroradiologist. We calculated the inter-class correlation coefficient (ICC) to evaluate the inter-observer reproducibility.

As the images were acquired from different CT scanners with different parameters, we performed data preprocessing before radiomics feature extraction (Lambin et al., 2017; Morin et al., 2018; Chen et al., 2021). Image resampling and gray-level discretization were used to reduce the variability of radiomics features (Shafiq-Ul-Hassan et al., 2018). A total of 1316 radiomics features were extracted from each ROI (see Supplementary Digital Content 3, which illustrates the possible pathophysiologic meaning of the features). All radiomics features were standardized by z-score to eliminate unit limits of each feature (Yang et al., 2019; see Supplementary Methods, Supplementary Digital Content 4, for further details about radiomics analysis).

Figure 1B indicates the feature selection procedure. Firstly, features with poor reproducibility (an ICC of <0.7) were excluded. After that, we used two different kinds of schemes to select informative radiomics features: (1) the minimum redundancy maximum relevance (mRMR) method was performed to rank the top 50 rupture-associated features while minimizing intra-feature correlation (Ding and Peng, 2005; Castiglioni et al., 2019), and then we used the least absolute shrinkage and selection operator (LASSO) method to select optimal features from those 50 features (Sauerbrei et al., 2007) (“mRMR-LASSO method”); (2) only LASSO method was used to selected optimal features (“LASSO method”) (see Supplementary Methods, Supplementary Digital Content 5, for further detailed information of mRMR and LASSO). We applied the logistics regression model to build radiomics signatures. Discrimination ability of the “mRMR-LASSO model” and “LASSO model” were compared. The features with better performance were used for subsequent analysis. Moreover, we calculated the Rad score through a linear combination of selected features by multiplying with their LASSO coefficients (Huang et al., 2016).

Machine Learning Models Development and Validation

Support vector machine (SVM) is a supervised machine learning method that classifies data points by maximizing the distance between classes in a high-dimensional space (Orru et al., 2012). We applied SVM with a 10-fold cross-validation to construct models. As shown in Figure 1D, optimal radiomics features were introduced into the radiomics model (R-model). The morphological predictors of aneurysm rupture were introduced into the morphological model (M-model). Optimal radiomics features and morphological features were put together to generate the radiomics-morphological model (RM-model). Models were trained using the training dataset and validated in the temporal and external validation datasets. Variance inflation factor (VIF) was used to detect multicollinearity of the enrolled features, and a VIF of ≥5 was considered as multicollinearity (Akinwande et al., 2015).

The model performance was evaluated using the receiver operating characteristic (ROC) curve. The DeLong test was used for comparisons of an area under the receiver operating curves (AUCs) of different models (DeLong et al., 1988). The decision curve analysis (DCA) was applied to assess the clinical utility of the models. Besides, net reclassification improvement (NRI) (Pencina et al., 2011) was calculated to evaluate the improvement in the discrimination ability of different models.

Nomogram Construction and Evaluation

To provide an easy and visualized rupture risk-scoring system, we constructed a comprehensive nomogram. The top five rupture-associated factors among the Rad score, clinical and morphological features were selected using the mRMR method. These five factors were used to generate the nomogram. The discrimination of the nomogram was assessed with ROC curves. The agreement between predicted rupture and observed rupture was evaluated using the calibration curve and the Hosmer–Lemeshow test (Kramer and Zimmerman, 2007). The discrimination and calibration of the nomogram were appraised in the training and validation datasets.

Statistical Analysis

Categorical variables are presented as counts (with percentages), while continuous variables are presented as medians [interquartile range, (IQR)]. We used Student t-tests or Mann–Whitney U tests to evaluate the differences in continuous variables. Differences in categorical variables were assessed using the χ² test or Fisher exact test (two-tailed). A p-value of <0.05 indicates a statistical difference. Statistical analysis and model construction were conducted using SPSS (version 24.0) and R (version 3.6.1).

Results

Clinical and Morphological Characteristics

A total of 632 patients with 668 MCA aneurysms (423 ruptured aneurysms) from five hospitals were included in our study. Multiple aneurysms were presented in 132 (20.9%) patients. Thirty-two of the 132 (24.2%) patients have bilateral MCA aneurysms. There were 407, 152, and 73 patients with 438, 155, and 75 aneurysms in the training, temporal validation, and external validation datasets. Table 1 shows the clinical and morphological characteristics of the training dataset (see Supplementary Digital Content 6, which shows characteristics of validation datasets). In univariate analysis, patients with ruptured aneurysms were younger and were less common to have a history of hypertension (p < 0.05).

TABLE 1

Table 1. Baseline characteristics of patients in the training dataset.

For morphological features, the mean ICC value of the nine morphological features was 0.924. The kappa value of the irregular shape and daughter dome is 0.603 and 0.838, respectively (p-value for all <0.001). Vessel size, aneurysm size, neck size, AR, SR, aneurysm height, perpendicular height, aneurysm angle, irregular shape, and daughter dome were associated with aneurysm rupture (p < 0.05). The result of multivariate analysis (Table 2) indicates that SR [odds ratio (OR), 1.607 (95% CI, 1.309, 1.973); p < 0.001], neck size [OR, 0.690 (95% CI, 0.596, 0.799), p < 0.001], multiplicity [OR, 0.389 (95% CI, 0.244, 0.621), p < 0.001], and daughter dome [OR, 2.987 (95% CI, 1.650, 5.406), p < 0.001] were independent predictors of MCA aneurysms rupture.

TABLE 2

Table 2. Univariate and multivariable analysis of morphological and clinical features associated with aneurysm rupture.

Optimal Radiomics Features Detection

The mean ICC value of the overall 1316 radiomics features was 0.751. Eight hundred and eighty-one radiomics features showed a high interobserver agreement (an ICC value of ≥0.7). As shown in Table 3, the “mRMR-LASSO model” presented an AUC of 0.767 and 0.828 in the training and temporal validation dataset, respectively. The “mRMR-LASSO model” presented a higher discrimination ability than the “LASSO model.” Therefore, those features selected using the “mRMR-LASSO method” were used for R-model construction. The Rad score was calculated (see Supplementary Results, Supplementary Digital Content 7, which indicates the Rad score calculation formula).

TABLE 3

Table 3. Performance of the “LASSO model” and “mRMR-LASSO model.”

Machine Learning Models Construction and Evaluation

The performance of the models was shown in Table 4 and Figure 2. The R-model obtained an AUC of 0.822 (95% CI, 0.776, 0.867) in the training dataset. In the temporal and external validation dataset, the R-model presented an AUC of 0.817 (95% CI, 0.744, 0.890) and 0.691 (95% CI, 0.567, 0.816), respectively. Four morphological predictors, including SR, neck size, multiplicity, and daughter dome, were used to construct the M-model. The M-model obtained an AUC of 0.798 (95% CI, 0.749, 0.846), 0.751 (95% CI, 0.674, 0.828), and 0.624 (95% CI, 0.490, 0.759) in the training, temporal, and external validation datasets, respectively. The p-values from the DeLong test of the statistical comparison of the ROC curves are given in Table 5.

TABLE 4

Table 4. Performance of the radiomics, morphological, radiomics-morphological, clinical-morphological, and clinical-radiomics-morphological models.

FIGURE 2

Figure 2. The receiver operating characteristic (ROC) curves, accuracy, sensitivity, and specificity of the radiomics model (R-model), morphological model (M-model), radiomics-morphological model (RM-model), clinical-morphological model (CM-model); and clinical-radiomics-morphological model (CRM-model). The ROC curves of the three models in training (A), temporal validation (B), and external validation datasets (C). The accuracy, sensitivity, and specificity of the three models in the training (D), temporal validation (E), and external validation datasets (F). ACC, accuracy; SEN, sensitivity; SPE, specificity.

TABLE 5

Table 5. The p-values of the DeLong test of the statistical comparison of the ROC curves in all datasets.

The optimal radiomics features and morphological predictors were enrolled in the RM-model. The RM-model exhibited good ability in classifying ruptured MCA aneurysms, with an AUC of 0.848 (95% CI, 0.810, 0.885), accuracy of 0.795, sensitivity of 0.788, and specificity of 0.809 in the training dataset. We further validated the RM-model in two validation datasets. The AUC of RM-model for ruptured MCA aneurysms classification was 0.865 (95% CI, 0.807, 0.924) and 0.721 (95% CI, 0.601, 0.841) in the temporal and external validation datasets, respectively (Figure 2 and Table 4). Multicollinearity was not observed between those selected radiomics features and morphological predictors (VIF for all <2).

Compared with the single R-model and M-model, the RM-model achieved a higher AUC. In the training dataset, the RM-model outperformed the M-model [AUC (95% CI), 0.848 (0.810, 0.885) vs. 0.798 (0.749, 0.846), p = 0.041]. In the validation datasets, the RM-model tended to have a better presented higher AUC than the single M-model. The difference was statistically significant in the temporal validation dataset (p < 0.005) while it was not statistically significant in the external validation dataset (p = 0.224).

We further added clinical features (hypertension, smoking, age, and sex) into the M-model and RM-model to construct the clinical-morphological model (CM-model) and clinical-radiomics-morphological model (CRM-model). As it was shown in Table 4 and Figure 2, the CRM-model obtained an AUC (95% CI) of 0.856 (0.820, 0.892), 0.882 (0.828, 0.936), and 0.738 (0.618, 0.857) in the three datasets, respectively. The CRM-model outperformed the CM-model in the training and temporal validation datasets (p < 0.05). However, the difference was not observed in the external validation dataset. We further calculated NRI to evaluate the improvement of discrimination by adding radiomics features to CM-model. We found that adding radiomics features to CM-mode improved the net reclassification indices in the three datasets (additive NRI, 52.40%, 89.66%, and 24.62%, respectively). This indicated that compared to the CM-model, the CRM-model correctly reclassified 52.40%, 89.66%, and 24.62% cases in the three datasets, respectively.

With respect to clinical utility, the DCA (Figure 3B) indicated that the RM-model had a higher overall net benefit in distinguishing ruptured aneurysms than the single R-model and M-model for most of the threshold probabilities. The CRM-model obtained the highest net benefit for most of the threshold probabilities. Combining radiomics features to conventional models resulted in an extra net-benefit compared with the M-model and CM-model.

FIGURE 3

Figure 3. The comprehensive nomogram for classifying ruptured MCA aneurysm and decision curve analysis in the overall patients. (A) The comprehensive nomogram for predicting aneurysm rupture. (B) Decision curve analysis in overall patients. The y-axis indicates the net benefit; the x-axis indicates threshold probability. The gray line represents the assumption that all aneurysms rupture. The black line represents the assumption that no aneurysm ruptures. The red line, green line, gold line, orange line, and blue line represent the net benefit of the radiomics model (R-model), morphological model (M-model), radiomics-morphological model (RM-model), clinical-morphological model (CM-model), and clinical-radiomics-morphological model (CRM-model), respectively. Calibration of the nomogram in the training (C), temporal validation (D), and external validation datasets (E).

Nomogram Construction and Evaluation

The shape, vessel size, AR, multiplicity, and Rad score were incorporated into the comprehensive nomogram (Figure 3A). The nomogram presented satisfying discrimination ability with an AUC of 0.771 (95% CI, 0.723, 0.818), 0.823, (95% CI, 0.757, 0.889), and 0.709 (95% CI, 0.592, 0.827) in the training, temporal validation, and external validation datasets, respectively. The calibration curve (Figures 3C–E) and the Hosmer–Lemeshow test (p = 0.731, 0.325, and 0.067, in the three datasets, respectively) indicate good calibration.

Discussion

This study developed and validated ruptured MCA aneurysms classification models based on radiomics and morphological features in a large and multicentral dataset (a total of 668 MCA aneurysms were enrolled). It was proved that robust radiomics features extracted from CTA images could classify ruptured MCA aneurysms. The RM-model could identify more than 74% ruptured MCA aneurysms with a specificity of 72–81%. Additionally, we provided an easy and visualized rupture risk-scoring system for MCA aneurysm patients through the nomogram.

In this study, a total of 1316 radiomics features were extracted for each aneurysm. Among those numerous radiomics features, the features indicating image heterogeneity (e.g., Dependence Non-Uniformity Normalized, Cluster Shade, and Variance) were screened out as the optimal signature for ruptured aneurysms classification. This is in agreement with the study of Ou et al. (2021). The possible explanation for this is that CTA image heterogeneity is caused by the contrast maldistribution in the vessel lumen. The uneven distribution of contrast indicates turbulent flow (George et al., 2016; Aghayev et al., 2018; Ou et al., 2021), which was one of the risk factors for aneurysms rupture (Lv et al., 2020). Therefore, we speculate that CTA-derived radiomics features might imply the hemodynamics condition of aneurysms.

Unlike former research (Liu et al., 2019; Ou et al., 2021), shape radiomics features were not selected for model construction. We exclude some shape features before the feature selection due to their poor interobserver agreement (e.g., the shape radiomics features Sphericity and Flatness obtained an ICC value of 0.536 and 0.590, respectively). Only highly stable features were used in the model construction procedure. The temporal and external validation datasets further verified the robustness and generalizability of the results.

It is generally acknowledged that vessel wall degradation and abnormal morphological condition are related to aneurysms’ rupture. We found that daughter dome, multiplicity, neck size, and SR were independent predictors for MCA aneurysm rupture, which have been reported by other researchers (Can et al., 2015; Zhang et al., 2019; Ou et al., 2020; Tanioka et al., 2020; Zhu et al., 2020).

Both radiomics and morphological features originate from CTA images, but their biological meanings and analysis procedures are different. Radiomics features describe the shape and texture characteristics of aneurysms from the micro point of view (Lambin et al., 2017; Xu et al., 2019), while the morphological features measure the macroscopic observation of the aneurysms. Radiomics features are three-dimensional derived, high-through biomarkers but lacking clinical interpretability. Meanwhile, those morphological features are two-dimensional measured features with relatively less information, but they are still fundamental tools for aneurysm evaluation. Though the statistically significant improvement in AUC of adding radiomics was not observed in the external dataset, considering its objectivity and its extra net-benefit in DCA, radiomics could be a possible choice in clinical practice, enabling better patient management.

Aneurysms rupture leads to catastrophic consequences. Preventive treatment for high rupture risk patients is necessary. However, preventive treatments for low rupture risk patients may cause more harm than good due to operation-related complications (Naggara et al., 2010). Therefore, except for stratifying rupture-prone aneurysms, accurate identification of aneurysms at a relatively low rupture risk is also essential. Compared to the CM-model (which is routinely used in clinical practice), the CRM-model obtained a higher specificity, positive predictive value (PPV), and the CRM-model correctly reclassified 52.40%, 89.66%, and 24.62% patients in the three datasets, respectively. This indicated that adding radiomics features to conventional models might not only classify ruptured aneurysms but also help in recognizing unruptured aneurysms, which may reduce unnecessary treatment for unruptured aneurysms patients.

There are some limitations in this study. First, this is a retrospective, cross-sectional study without the longitudinal follow-up of aneurysms, which might inherently cause biases. Second, the aneurysms’ morphological changes after rupture were not considered because it is hard to collect the morphological change before and after the aneurysm rupture due to ethical issues. Third, most patients with SAH history were excluded because of their surgery experience. This could cause potential selection bias. Forth, images from external validation datasets contain different scanning protocols, which might affect the validation results. We performed image resampling and gray-level discretization to reduce the variability of images. Moreover, samples from the other four hospitals were merged into one external validation dataset. The good results further indicated the robustness of the models.

Conclusion

In conclusion, we analyzed the MCA aneurysms rupture by clinical, radiomics, and morphological features using multicentral data. We answered two critical questions: (1) Robust radiomics features could classify ruptured MCA aneurysms. (2) The integration of radiomics into conventional clinical and morphological models might provide additional benefit in ruptured MCA aneurysms classification. An easy and visualized rupture risk-scoring nomogram was generated. This may aid in the rupture-risk assessment of MCA aneurysms.

Data Availability Statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author/s.

Ethics Statement

The studies involving human participants were reviewed and approved by the Medical Ethics Committee of the First Affiliated Hospital of Wenzhou Medical University. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

Author Contributions

DZ wrote the manuscript. DZ, PP, and YC interpreted the data and prepared the tables and figures. JZ, YC, and KZ revised the manuscript for intellectual content. CC, QL, JZ, NX, HW, BL, YN, and XJ acquired the data. YY contributed to the conception and the design of the study. All co-authors read and revised the manuscript.

Funding

This study was supported by the Wenzhou Major Program of Science and Technology Innovation (Grant No. ZY2020012), the Health Foundation for Creative Talents in Zhejiang Province, China (Grant No. 2016), the Project Foundation for the College Young and Middle-aged Academic Leader of Zhejiang Province, China (Grant No. 2017), and Basic Research Project of Wenzhou (Grant No. Y2020164).

Conflict of Interest

PP was employed by the company GE Healthcare China Co., Ltd., Shanghai.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnins.2021.721268/full#supplementary-material

References

Aghayev, A., Giannopoulos, A. A., Gronsbell, J., George, E., Cai, T., Steigner, M. L., et al. (2018). Common First-Pass CT Angiography Findings Associated With Rapid Growth Rate in Abdominal Aorta Aneurysms Between 3 and 5 cm in Largest Diameter. AJR Am. J. Roentgenol. 210, 431–437. doi: 10.2214/AJR.17.18094

PubMed Abstract | CrossRef Full Text | Google Scholar

Akinwande, M. O., Dikko, H. G., and Samson, A. (2015). Variance Inflation Factor: As a Condition for the Inclusion of Suppressor Variable(s) in Regression Analysis. Open J. Stat. 05, 754–767. doi: 10.4236/ojs.2015.57075

CrossRef Full Text | Google Scholar

Al-Shahi Salman, R., Frantzias, J., Lee, R. J., Lyden, P. D., Battey, T. W. K., Ayres, A. M., et al. (2018). Absolute risk and predictors of the growth of acute spontaneous intracerebral haemorrhage: a systematic review and meta-analysis of individual patient data. Lancet Neurol. 17, 885–894. doi: 10.1016/S1474-4422(18)30253-9

CrossRef Full Text | Google Scholar

Can, A., Ho, A. L., Dammers, R., Dirven, C. M., and Du, R. (2015). Morphological parameters associated with middle cerebral artery aneurysms. Neurosurgery 76, 721–726. doi: 10.1227/neu.0000000000000713

PubMed Abstract | CrossRef Full Text | Google Scholar

Castiglioni, I., Gallivanone, F., Soda, P., Avanzo, M., Stancanello, J., Aiello, M., et al. (2019). AI-based applications in hybrid imaging: how to build smart and truly multi-parametric decision models for radiomics. Eur. J. Nucl. Med. Mol. Imaging 46, 2673–2699. doi: 10.1007/s00259-019-04414-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Chen, Q., Xia, T., Zhang, M., Xia, N., Liu, J., and Yang, Y. (2021). Radiomics in Stroke Neuroimaging: Techniques, Applications, and Challenges. Aging Dis 12, 143–154. doi: 10.14336/AD.2020.0421

PubMed Abstract | CrossRef Full Text | Google Scholar

Chen, Y., Lin, B., Zhou, J., Chen, L., Yang, Y., and Zhao, B. (2020). Morphological predictors of middle cerebral artery bifurcation aneurysm rupture. Clin. Neurol. Neurosurg. 192:105708. doi: 10.1016/j.clineuro.2020.105708

PubMed Abstract | CrossRef Full Text | Google Scholar

Choe, J., Lee, S. M., Do, K. H., Lee, G., Lee, J. G., Lee, S. M., et al. (2019). Deep Learning-based Image Conversion of CT Reconstruction Kernels Improves Radiomics Reproducibility for Pulmonary Nodules or Masses. Radiology 292, 365–373. doi: 10.1148/radiol.2019181960 PubMed

CrossRef Full Text | PubMed Abstract | Google Scholar

Collins, G. S., Reitsma, J. B., Altman, D. G., and Moons, K. G. M. (2015). members of the Tg. Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD): The TRIPOD Statement. Eur. Urol. 67, 1142–1151. doi: 10.1016/j.eururo.2014.11.025

PubMed Abstract | CrossRef Full Text | Google Scholar

DeLong, E. R., DeLong, D. M., and Clarke-Pearson, D. L. (1988). Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44, 837–845. doi: 10.2307/2531595

CrossRef Full Text | Google Scholar

Ding, C., and Peng, H. (2005). Minimum redundancy feature selection from microarray gene expression data. J. Bioinform. Comput. Biol. 3, 185–205. doi: 10.1142/s0219720005001004

PubMed Abstract | CrossRef Full Text | Google Scholar

George, E., Giannopoulos, A. A., Aghayev, A., Rohatgi, S., Imanzadeh, A., Antoniadis, A. P., et al. (2016). Contrast inhomogeneity in CT angiography of the abdominal aortic aneurysm. J. Cardiovasc. Comput. Tomogr. 10, 179–183. doi: 10.1016/j.jcct.2015.11.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Greving, J. P., Wermer, M. J., Brown, R. D. Jr., Morita, A., Juvela, S., Yonekura, M., et al. (2014). Development of the PHASES score for prediction of risk of rupture of intracranial aneurysms: a pooled analysis of six prospective cohort studies. Lancet Neurol. 13, 59–66. doi: 10.1016/S1474-4422(13)70263-1

CrossRef Full Text | Google Scholar

Hua, W., Xiao, T., Jiang, X., Liu, Z., Wang, M., Zheng, H., et al. (2020). Lymph-vascular space invasion prediction in cervical cancer: Exploring radiomics and deep learning multilevel features of tumor and peritumor tissue on multiparametric MRI. Biomedical. Signal Proc. Control 58:101869. doi: 10.1016/j.bspc.2020.101869

CrossRef Full Text | Google Scholar

Huang, Y. Q., Liang, C. H., He, L., Tian, J., Liang, C. S., Chen, X., et al. (2016). Development and Validation of a Radiomics Nomogram for Preoperative Prediction of Lymph Node Metastasis in Colorectal Cancer. J. Clin. Oncol. 34, 2157–2164. doi: 10.1200/JCO.2015.65.9128

PubMed Abstract | CrossRef Full Text | Google Scholar

Huttunen, T., von und zu Fraunberg, M., Frosen, J., Lehecka, M., Tromp, G., Helin, K., et al. (2010). Saccular intracranial aneurysm disease: distribution of site, size, and age suggests different etiologies for aneurysm formation and rupture in 316 familial and 1454 sporadic eastern Finnish patients. Neurosurgery 66, 631–638.

Google Scholar

Korja, M., Lehto, H., and Juvela, S. (2014). Lifelong rupture risk of intracranial aneurysms depends on risk factors: a prospective Finnish cohort study. Stroke 45, 1958–1963. doi: 10.1161/STROKEAHA.114.005318

PubMed Abstract | CrossRef Full Text | Google Scholar

Kramer, A. A., and Zimmerman, J. E. (2007). Assessing the calibration of mortality benchmarks in critical care: The Hosmer-Lemeshow test revisited. Crit. Care Med. 35, 2052–2056. doi: 10.1097/01.CCM.0000275267.64078.B0

CrossRef Full Text | Google Scholar

Lambin, P., Leijenaar, R. T. H., Deist, T. M., Peerlings, J., de Jong, E. E. C., van Timmeren, J., et al. (2017). Radiomics: the bridge between medical imaging and personalized medicine. Nat. Rev. Clin. Oncol. 14, 749–762. doi: 10.1038/nrclinonc.2017.141

PubMed Abstract | CrossRef Full Text | Google Scholar

Lindgren, A. E., Koivisto, T., Bjorkman, J., von Und Zu Fraunberg, M., Helin, K., and Jaaskelainen, J. E. (2016). Irregular Shape of Intracranial Aneurysm Indicates Rupture Risk Irrespective of Size in a Population-Based Cohort. Stroke 47, 1219–1226. doi: 10.1161/strokeaha.115.012404

PubMed Abstract | CrossRef Full Text | Google Scholar

Liu, Q., Jiang, P., Jiang, Y., Ge, H., Li, S., Jin, H., et al. (2019). Prediction of Aneurysm Stability Using a Machine Learning Model Based on PyRadiomics-Derived Morphological Features. Stroke 50, 2314–2321. doi: 10.1161/STROKEAHA.119.025777

PubMed Abstract | CrossRef Full Text | Google Scholar

Lv, N., Karmonik, C., Chen, S., Wang, X., Fang, Y., Huang, Q., et al. (2020). Wall Enhancement, Hemodynamics, and Morphology in Unruptured Intracranial Aneurysms with High Rupture Risk. Transl. Stroke Res. 11, 882–889. doi: 10.1007/s12975-020-00782-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Mackin, D., Fave, X., Zhang, L., Fried, D., Yang, J., Taylor, B., et al. (2015). Measuring Computed Tomography Scanner Variability of Radiomics Features. Invest. Radiol. 50, 757–765. doi: 10.1097/RLI.0000000000000180

PubMed Abstract | CrossRef Full Text | Google Scholar

Morin, O., Vallieres, M., Jochems, A., Woodruff, H. C., Valdes, G., Braunstein, S. E., et al. (2018). A Deep Look Into the Future of Quantitative Imaging in Oncology: A Statement of Working Principles and Proposal for Change. Int. J. Radiat. Oncol. Biol. Phys. 102, 1074–1082. doi: 10.1016/j.ijrobp.2018.08.032

PubMed Abstract | CrossRef Full Text | Google Scholar

Naggara, O. N., White, P. M., Guilbert, F., Roy, D., Weill, A., and Raymond, J. (2010). Endovascular treatment of intracranial unruptured aneurysms: systematic review and meta-analysis of the literature on safety and efficacy. Radiology 256, 887–897. doi: 10.1148/radiol.10091982 PubMed

CrossRef Full Text | PubMed Abstract | Google Scholar

Orru, G., Pettersson-Yeo, W., Marquand, A. F., Sartori, G., and Mechelli, A. (2012). Using Support Vector Machine to identify imaging biomarkers of neurological and psychiatric disease: a critical review. Neurosci Biobehav Rev 36, 1140–1152. doi: 10.1016/j.neubiorev.2012.01.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Ou, C., Chong, W., Duan, C. Z., Zhang, X., Morgan, M., and Qian, Y. A. (2021). preliminary investigation of radiomics differences between ruptured and unruptured intracranial aneurysms. Eur. Radiol. 31, 2716–2725. doi: 10.1007/s00330-020-07325-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Ou, C., Liu, J., Qian, Y., Chong, W., Zhang, X., Liu, W., et al. (2020). Rupture Risk Assessment for Cerebral Aneurysm Using Interpretable Machine Learning on Multidimensional Data. Front Neurol 11:570181. doi: 10.3389/fneur.2020.570181

PubMed Abstract | CrossRef Full Text | Google Scholar

Pencina, M. J., D’Agostino, R. B. Sr., and Steyerberg, E. W. (2011). Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers. Stat. Med. 30, 11–21. doi: 10.1002/sim.4085

PubMed Abstract | CrossRef Full Text | Google Scholar

Sauerbrei, W., Royston, P., and Binder, H. (2007). Selection of important variables and determination of functional form for continuous predictors in multivariable model building. Stat. Med. 26, 5512–5528. doi: 10.1002/sim.3148

PubMed Abstract | CrossRef Full Text | Google Scholar

Shafiq-Ul-Hassan, M., Latifi, K., Zhang, G., Ullah, G., Gillies, R., and Moros, E. (2018). Voxel size and gray level normalization of CT radiomic features in lung cancer. Sci. Rep. 8:10545. doi: 10.1038/s41598-018-28895-9

PubMed Abstract | CrossRef Full Text | Google Scholar

Shi, Z., Chen, G. Z., Mao, L., Li, X. L., Zhou, C. S., Xia, S., et al. (2021). Machine Learning-Based Prediction of Small Intracranial Aneurysm Rupture Status Using CTA-Derived Hemodynamics: A Multicenter Study. AJNR Am. J. Neuroradiol. 2021:7034. doi: 10.3174/ajnr.A7034

PubMed Abstract | CrossRef Full Text | Google Scholar

Tanioka, S., Ishida, F., Yamamoto, A., Shimizu, S., Sakaida, H., Toyoda, M., et al. (2020). Machine Learning Classification of Cerebral Aneurysm Rupture Status with Morphologic Variables and Hemodynamic Parameters. Radiol. Artif Intell 2:e190077. doi: 10.1148/ryai.2019190077

PubMed Abstract | CrossRef Full Text | Google Scholar

Tomaszewski, M. R., and Gillies, R. J. (2021). The Biological Meaning of Radiomic Features. Radiology 298, 505–516. doi: 10.1148/radiol.2021202553

PubMed Abstract | CrossRef Full Text | Google Scholar

Vlak, M. H., Algra, A., Brandenburg, R., and Rinkel, G. J. (2011). Prevalence of unruptured intracranial aneurysms, with emphasis on sex, age, comorbidity, country, and time period: a systematic review and meta-analysis. Lancet Neurol. 10, 626–636. doi: 10.1016/S1474-4422(11)70109-0

CrossRef Full Text | Google Scholar

Xu, X., Zhang, H. L., Liu, Q. P., Sun, S. W., Zhang, J., Zhu, F. P., et al. (2019). Radiomic analysis of contrast-enhanced CT predicts microvascular invasion and outcome in hepatocellular carcinoma. J. Hepatol. 70, 1133–1144. doi: 10.1016/j.jhep.2019.02.023

PubMed Abstract | CrossRef Full Text | Google Scholar

Yang, L., Gu, D., Wei, J., Yang, C., Rao, S., Wang, W., et al. (2019). A Radiomics Nomogram for Preoperative Prediction of Microvascular Invasion in Hepatocellular Carcinoma. Liver Cancer 8, 373–386. doi: 10.1159/000494099

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhang, J., Can, A., Mukundan, S., Steigner, M., Castro, V. M., Dligach, D., et al. (2019). Morphological Variables Associated With Ruptured Middle Cerebral Artery Aneurysms. Neurosurgery 85, 75–83. doi: 10.1093/neuros/nyy213 PubMed

CrossRef Full Text | PubMed Abstract | Google Scholar

Zhou, Y., Xu, J., Liu, Q., Li, C., Liu, Z., Wang, M., et al. (2018). A Radiomics Approach With CNN for Shear-Wave Elastography Breast Tumor Classification. IEEE Trans. Biomed. Eng. 65, 1935–1942. doi: 10.1109/TBME.2018.2844188

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhu, D., Zhang, M., Li, Q., Liu, J., Zhuang, Y., Chen, Q., et al. (2021). Can perihaematomal radiomics features predict haematoma expansion? Clin. Radiol. 2021:003. doi: 10.1016/j.crad.2021.03.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhu, D. Q., Chen, Q., Xiang, Y. L., Zhan, C. Y., Zhang, M. Y., Chen, C., et al. (2021). Predicting intraventricular hemorrhage growth with a machine learning-based, radiomics-clinical model. Aging 13, 12833–12848. doi: 10.18632/aging.202954

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhu, W., Li, W., Tian, Z., Zhang, Y., Wang, K., Zhang, Y., et al. (2020). Stability Assessment of Intracranial Aneurysms Using Machine Learning Based on Clinical and Morphological Features. Transl. Stroke Res. 11, 1287–1295. doi: 10.1007/s12975-020-00811-2

PubMed Abstract | CrossRef Full Text | Google Scholar