Jinfen Kong*
Danfen Zhang- Department of Radiology, Yuhuan Second People's Hospital, Yuhuan, Taizhou, Zhejiang, China
Background: Pre-treatment prediction of reperfusion and long-term prognosis in acute ischemic stroke (AIS) patients is crucial for effective treatment and decision-making. Recent studies have demonstrated that the inclusion of radiomics data can improve the performance of predictive models. This paper reviews published studies focused on radiomics-based prediction of reperfusion and long-term prognosis in AIS patients.
Methods: We systematically searched PubMed, Web of Science, and Cochrane databases up to September 9, 2023, for studies on radiomics-based prediction of AIS patient outcomes. The methodological quality of the included studies was evaluated using the phase classification criteria, the radiomics quality scoring (RQS) tool, and the Prediction model Risk Of Bias Assessment Tool (PROBAST). Two separate meta-analyses were performed of these studies that predict long-term prognosis and reperfusion in AIS patients.
Results: Sixteen studies with sample sizes ranging from 67 to 3,001 were identified. Ten studies were classified as phase II, and the remaining were categorized as phase 0 (n = 2), phase I (n = 1), and phase III (n = 3). The mean RQS score of all studies was 39.41%, ranging from 5.56 to 75%. Most studies (87.5%, 14/16) were at high risk of bias due to their retrospective design. The remaining two studies were categorized as low risk and unclear risk, respectively. The pooled area under the curve (AUC) was 0.88 [95% confidence interval (CI) 0.84–0.92] for predicting the long-term prognosis and 0.80 (95% CI 0.74–0.86) for predicting reperfusion in AIS.
Conclusion: Radiomics has the potential to predict immediate reperfusion and long-term outcomes in AIS patients. Further external validation and evaluation within the clinical workflow can facilitate personalized treatment for AIS patients. This systematic review provides valuable insights for optimizing radiomics prediction systems for both reperfusion and long-term outcomes in AIS patients.
Systematic review registration: https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42023461671, identifier CRD42023461671.
1 Introduction
Stroke is a major global health issue, characterized by high disability, fatality, and recurrence rates (1). It brings a heavy burden on individuals, families, and society (2). Despite the decrease in incidence and prevalence of stroke globally, China still has a rising number of new cases, with 250 million reported every year (3–5). Acute ischemic stroke (AIS), accounts for 60%-80% of strokes (6), as defined by sudden neurological deficits caused by focal cerebral ischemia (7). Early intervention is critical to improve outcomes in AIS patients. The goal of treatment is to achieve recanalization and reperfusion of the ischaemic penumbra (8, 9). However, many individuals still have severe residual impairments after treatments. Therefore, it is important to accurately predict the prognosis of stroke and determine the long-term treatment plan for patients (8, 10).
Rapid and accurate prediction of outcomes such as success reperfusion and long-term prognosis is vital for effective treatment selection. Traditional predictive models often rely on clinical and conventional image parameters, but their predictive power has limitations. Radiomics is a rapidly evolving field of research that involves extracting quantitative metrics, known as radiomic features, from medical images (11–13). Computed tomography (CT) and magnetic resonance imaging (MRI) are primary diagnostic tools (14, 15), while their radiomic features also hold great potential for prognosis prediction (16, 17). Given the complexity of radiomics data, machine learning (ML) algorithms play a vital role in its application (18–21), particularly in acute stroke imaging (22, 23).
In the landscape of AIS management, conventional imaging remains a cornerstone in diagnosis. The American Heart Association/American Stroke Association (AHA/ASA) guidelines for the early management of patients with AIS and the Chinese guidelines for the diagnosis and treatment of AIS both include non-contrast CT and conventional MRI as an important part of stroke diagnosis (24, 25). Additionally, techniques like CT perfusion and multimodal MRI are invaluable in guiding treatments such as thrombolytic therapy and endovascular thrombolysis (26–29). However, the rapidly developing field of radiomics is providing new pathways for diagnosis and treatment decision-making. For instance, a recent study illustrated that high-dimension radiomics features derived from CT angiography enhances the accuracy of AIS diagnosis and subtype classification, which is crucial for timely intervention (30). Additionally, another study has shown that radiomic models based on apparent diffusion coefficient (ADC) map are highly effective in identifying the ischemic penumbra, which is important in assessing the normal and ischemic penumbra areas and influencing treatment decisions (31). In terms of prognostic prediction, radiomics studies have seen substantial growth in recent years. Since prognosis in AIS is influenced by various factors, these studies often aim to combine radiomics features with clinical information to more precise predictions of patient outcomes. Although currently not used in clinical practice, the combination of radiomics with established imaging techniques will allow us to take a significant step forward in understanding and processing AIS.
While several studies have evaluated the utility of radiomics in AIS prediction, there is a lack of comprehensive reviews that systematically evaluate the methodological quality and predictive accuracy of these studies. This systematic review and meta-analysis aim to address this gap, focusing on reperfusion and long-term prognosis prediction in AIS patients.
2 Methods
This systematic review and meta-analysis strictly adhered to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses 2020 (PRISMA 2020) statement. The PRISMA checklist is provided in Supplementary Table 1. And the protocol is prospectively registered in PROSPERO (CRD4202346167).
2.1 Literature search strategy
A comprehensive literature search for potentially relevant articles was conducted in PubMed, Cochorane, and Web of Science databases (WOS), with data retrieval up to September 9, 2023. A researcher (Jinfen Kong) designed the keywords and search strategy of this systematic review, and both subject headings and free words were searched. The complete search strategy can be found in the Supplementary Table 2.
2.2 Inclusion and exclusion criteria
2.2.1 Inclusion criteria
(1) Patients were diagnosed with AIS using CT or MRI.
(2) Radiomic features were utilized to predict outcome in AIS patients.
(3) Long-term outcome was measured using the modified Rankin Scale (mRS) or discharge National Institute of Health Stroke Scale (NIHSS).
(4) Reperfusion was assessed using the modified Thrombolysis in Cerebral Infarction (mTICI) scale or other relevant methods.
(5) Study participants were aged 18 and older.
(6) Full-text was available and articles were written in English.
2.2.2 Exclusion criteria
(1) Studies conducted in phantom or animal models.
(2) Case reports or small case series (≤10 patients).
(3) Reviews, poster presentations, letters, or meeting abstracts.
(4) Cerebral hemorrhage resulting from secondary causes such as cerebral trauma or subarachnoid hemorrhage.
(5) Predictive models rely solely on clinical factors without incorporating radiomic features.
2.3 Data extraction
All of the retrieved studies were managed using Endnote. After the removal of duplicates through automated and manual processes, two researchers (Jinfen Kong and Danfen Zhang) independently assessed the remaining articles. Preliminary screening was conducted based on titles and abstracts before downloading full texts. Eligible studies that met inclusion criteria were selected after a thorough examination of the full texts.
Prior to data extraction, a standardized data extraction sheet was prepared, including data source, sample size, population, study design, image modality, research question, treatment, software, segmentation, clinical and image features, validation approach, endpoints, reference standard, and classifier model types. Endpoints of interest were long-term prognosis and reperfusion. Long-term prognosis was defined as mRS or discharge NIHSS. Reperfusion was defined as mTICI or other methods. For long-term prognosis, mRS thresholds and follow-up time were recorded. For reperfusion, mTICI thresholds defined as successful reperfusion were recorded. Performance metrics area under the curve (AUC) were extracted.
2.4 Quality assessment
The methodological quality of the included studies was independently evaluated by two reviewers (Jinfen Kong and Danfen Zhang) and cross-checked for consistency.
2.4.1 Model quality assessment
The phase classification criteria is a model quality assessment tool (32). The parameters for phase categorization included sample size (<100 or >100), study design (retrospective or prospective), type of validation approach (internal or independent), and the development stage (pre- or post-marketing). The phase classification criteria categorized image mining studies into the discovery science and phases 0–IV.
2.4.2 Radiomics quality assessment
Radiomics quality was assessed using the 16-component Radiomics Quality Score (RQS) tool (33). Each study was assigned a number of points per RQS component and summed to give a total score ranging from −8 to +36. A score of −8 to 0 points corresponds to 0% and 36 points correspond to 100%. The mean score of the two evaluations is presented as a percentage.
2.4.3 Risk of bias assessment
The Risk of Bias (ROB) of included studies was evaluated using the Prediction Model Risk of Bias Assessment Tool (PROBAST) (34). This assessment encompassed four major domains: participants, predictors, outcomes, and statistical analysis, ultimately reflecting overall ROB and applicability. These domains include two, three, six and nine questions, respectively. Questions are answered as either yes/probable yes (Y/PY), no/probably no (N/PN), or no information (NI). If a domain is answered with at least a N/PN, it is considered at high ROB. An overall low ROB rating was achieved only when all four domains were rated as low ROB.
2.5 Meta-analysis
Two meta-analyses were performed: (1) A meta-analysis to evaluate the use of radiomics for predicting long-term prognosis, and (2) A meta-analysis to assess the use of radiomics in predicting reperfusion. When multiple models were reported in a study, only the one with the highest AUC was extracted. If multiple validation datasets of the optimal model were reported, we also extracted the highest AUC among them. Meta-analysis was performed on the metrics (AUC) for evaluating machine learning models. If AUC lacked 95% confidence interval (CI) and standard error (SE), we estimated SE based on a study by Debray et al. (35). Given the difference in variables and parameters in ML models, a random-effects model was employed to perform the meta-analysis. R package metafor (v4.4-0) was used for meta-analyses. Heterogeneity was assessed using the I2 statistic, with low, moderate, and high levels of heterogeneity corresponding to I2 values of 25, 50, and 75%, respectively (36). Publication bias was examined using funnel plots, Egger's bias test, and Begg's test. Two-sided p < 0.05 were considered statistically significant.
2.6 Sensitivity analysis
The impact of each study on the overall results was analyzed by excluding one study at a time and recalculating the combined effect size. R package metafor (v4.4-0) was used for sensitivity analysis.
3 Results
3.1 Study selection
Figure 1 illustrates the PRISMA flowchart outlining the study selection process. Our search strategy initially identified 87 studies from PubMed, 7 from Web of Science (WOS), and 7 from Cochrane (Figure 1 and Supplementary Table 2). After removing 6 duplicates, we screened 95 titles/abstracts. This led to the inclusion of 52 studies and 8 additional cited studies for full-text review. Ultimately, 16 peer-reviewed articles were included in this meta-analysis.
Figure 1. PRISMA flowchart of included studies.
3.2 Study characteristics
A total of 16 eligible studies were included in this systematic review (13, 37–51), and their characteristics are presented in Table 1 and Supplementary Table 3. The radiomics modality used in these studies was relatively balanced, with half employing CT image features and the remaining half using MR image features. The majority of studies used baseline images of AIS patients, while two studies (39, 46) included post-treatment images to predict long-term outcome. The treatment strategy in most studies is mechanical thrombectomy (MTB), while the remaining studies employed diverse treatments, including intravenous thrombolytic (IVT) therapy, endovascular treatment (EVT), intravenous recombinant tissue plasminogen activator (IV-rtPA) therapy, and other conventional treatment.
Table 1. Included radiomic studies.
Within these studies, ten are dedicated to predicting long-term prognosis, five focus on reperfusion (37, 40, 41, 43, 51), and one addresses both aspects (42). For long-term prognosis, assessment of the mRS spanned a follow-up period of 3–12 months, while assessments of the NIHSS occurred at discharge or on day 7. Most studies predicting long-term prognosis using mRS employed a threshold of 2 (Table 2). Regarding reperfusion, five out of six studies utilized the mTICI scale as the reference standard, employing thresholds of mTICI ≥ 2b, 2c, or 3 (Table 3).
Table 2. Patient follow-up time and stratification cut-off value in long-term prognostic modeling studies.
Table 3. Threshold for successful reperfusion.
Among the 16 studies, 12 conducted a combined model that integrated both clinical variables and radiomics features. Notably, 11 of these studies performed a comparation between models using only clinical features and models incorporating radiomics features. While 4 of these studies (41–43, 46) did not observe a significant difference in predictive performance, the remaining 7 (13, 38, 39, 44, 47–49) demonstrated that the inclusion of radiomics significantly improved the predictive ability for AIS prognosis. Moreover, among these 7 studies, 3 (38, 39, 44) directly compared the performance of the radiomics model with the clinical model, and all found that the radiomics model was superior. Additionally, one study (13) included conventional MRI factors like lesion optical densities (ODs), Fazekas scores, and admission DWI-ASPECTS, alongside clinical features. In this case, the AUC of the radiomics model was better than both the pure clinical model and the clinical + conventional MRI model.
The included studies were published between 2018 and 2023, with seven of them all published in 2023, indicating a huge explosion over these years. This trend highlights the increasing importance of radiomics in the field of prognostic prediction for patients with AIS.
3.3 Quality analysis
The sample size of the included studies ranged from 67 to 3,001, with thirteen studies (81.3%) enrolling more than 100 patients. Most studies were retrospective (81.3%), while one study (42) was prospective, and two studies (37, 44) employed a combination of retrospective training and prospective validation. Validation analysis was conducted in all studies, with seven studies using external datasets, seven studies employing the cross-validation method, and only three relying solely on internal datasets. According to the phase classification criteria, ten studies (62.5%) were classified as phase II, and the remaining were distributed as phase 0 (n = 2), phase I (n = 1), and phase III (n = 3) (Table 4).
Table 4. Summary of the quality of included studies.
In terms of radiomic-specific quality, Table 4 and Supplementary Table 4 presents the RQS scores of all included studies. And the individual score for each RQS items, for each of our two evaluators are shown in Supplementary Table 5 and Supplementary Figure 1. The mean score across all 16 studies was 39.41% (range 5.56–75%). Most studies scored within 30–40%, with only one study receiving a quality score of <10% (46) (Figures 2A, B). Most studies reported well-documented image acquisition protocols, nine studies (56.3%) mentioned that image segmentation was performed by 2 or more physicians. None addressed inter-scanner differences or vendor-dependent features. All studies acquired images from baseline scans except one study (46) used post-operative images, and another one (39) used both pre- and post-operative images. Feature dimension reduction was performed in most studies, except for one (46). Ten (62.5%) studies incorporated clinical features into radiomic models, and three of these studies suggested that this integration improved predictive performance. Clinical characteristics included in models were age, gender, age, sex, baseline NIHSS, baseline mRS, dyslipidemia, penetrating artery infarction, hypertension, previous stroke, ASPECTS, time since stroke (TSS), time to treatment (TTT), and more. The correlation between clinical factors and radiomic features was discussed in six (37.5%) studies. Only four studies conducted cut-off analysis using nomograms to assess risks. For model assessment, discrimination statistics were typically provided, whereas calibration statistics were less mentioned. Validation of radiomics signatures was performed in all studies, with 7 studies (43.8%) employing external datasets, and 7 using the cross-validation method. However, only one study (42) was prospectively registered. Regarding the clinical utility, 7 studies compared their models with the gold standard, indicating potential clinical utility. But none performed cost-effectiveness analysis. In terms of open science and data, two studies (42, 48) obtained radiomic features on a set of representative regions of interest (ROIs), with the calculated features and code being open access (Figures 2A, B).
Figure 2. Quality assessment of included studies. (A) Bar chart of the mean score of each study according to the radiomics quality scoring tool. (B) Pie chart of the mean score of studies according to the radiomics quality scoring tool. (C) Evaluation of Risk of bias.
For risk of bias (ROB), there are 13 of 16 models originated from retrospective studies, which carried a high risk of bias (ROB). The remaining three studies employed prospective validation, resulting in a low ROB (Table 4 and Supplementary Table 6). In terms of the assessment of predictors domain, 13 studies were rated at high ROB due to the potential bias in retrospective studies where researchers knew both predictors and data results. For the outcome domain, 14 studies had a low ROB, while one model had an unclear ROB (43), and another one had a high ROB (46). Lastly, as for the analysis domain, one study (37) had a high ROB because it selected predictors based solely on univariable analysis, and four studies (38, 40, 44, 46) had unclear ROB due to incomplete disclosure of how missing data were handled, failure to indicate whether data complexity was considered, lack of explanation on how overfitting was avoided, or not illustrating the weight of factors. The summary of ROB evaluation is presented in Figure 2C.
3.4 The value of radiomics in predicting AIS outcome
To evaluate the prognostic value of radiomics in AIS patients, we conducted two separate meta-analyses. The first meta-analysis focused on models designed to predict long-term prognosis in AIS patients, involving 14 radiomics models extracted from 11 studies. From these studies, we selected the best-performing model from 10 of them. Additionally, one study (45) implemented a distinctive approach by stratifying the mRS into 0, 1, 2, 3, and 4 and constructing an individualized model for each mRS score, where the model with mRS = 4 had a SE = 0 and was therefore excluded. Consequently, a total of 14 models were incorporated into the first meta-analysis. The result showed a pooled AUC of 0.883, calculated using a random-effects model, with a 95% CI ranging from 0.844 to 0.921 (p-value = 0.01) (Figure 3A). The I2 statistic is 61.8% which indicates high heterogeneity among these studies.
Figure 3. Forest plots of the predictive performance of radiomics models. Forest plots showed the predictive performance of radiomics models in (A) long-term prognosis (mRS) and (B) immediate reperfusion (mTICI) of AIS patients. For long-term prognosis, the results of subgroup analysis according to single-center or multicenter factors are shown; Area under curve (AUC) for each study is presented as a black dot, with the horizontal line indicating the 95% confidence interval (CI). The pooled result for all studies is presented as a black diamond.
To investigate the potential source of heterogeneity, we performed subgroup analysis, and found that multi-center or single-center is a significant contributor to the overall heterogeneity within the dataset. When analyzing the single-center subgroup, the pooled AUC was 0.95 (95% CI 0.91–0.98, p < 0.0001), and the I2 value was reduced to 47.31%. When analyzing the multi-center subgroup, the pooled AUC was 0.86 (95% CI 0.83–0.89, p < 0.0001), and the I2 value was markedly reduced to 6.39%, indicating a relatively lower heterogeneity (Figure 3A).
In the second meta-analysis, which included six radiomic studies that focused on reperfusion prediction, the pooled AUC was 0.800 (95%CI 0.734–0.863, p < 0.001) (Figure 3B). And the I2 statistic also suggested considerable heterogeneity in the AUC (I2 = 86.0%). We investigated potential factors that might contribute to the heterogeneity. However, variables such as multi-center or single-center, imaging modality, experimental design, location, and treatment were not able to explain the heterogeneity very well.
In conclusion, these results showed that the synthesized AUCs of the radiomics models are notably high. While some unexplained heterogeneity remains, radiomics exhibits promise in predicting long-term prognosis and reperfusion outcomes in AIS patients.
3.5 Publication bias
In the meta-analysis of AUC, no publication bias was detected for predicting both long-term outcome and reperfusion after stroke (Figures 4A, B). Begg's test results further confirmed this observation, with p-value = 0.25 for the prediction of long-term outcomes and p-value = 0.85 for the prediction of reperfusion.
Figure 4. Funnel plot for publication bias of AUC. Funnel plot showed the publication bias of AUC in (A) long-term prognosis (mRS) and (B) immediate reperfusion (mTICI) of AIS patients.
3.6 Robustness of meta-analysis
To demonstrate the robustness of meta-analysis, we performed sensitivity analyses by excluding individual study at a time and recalculating the combined AUC. For long-term prognosis prediction, the synthesized AUC values ranged from 0.832 to 0.949, with standard errors ranged from 0.0036 to 0.0052, which is closely matching the overall values (Table 5). For reperfusion prediction, the synthesized AUC values ranged from 0.743 to 0.847, and standard errors ranged from 0.0006 to 0.0035, which was also close to the overall values (Table 6). These results suggest that these two meta-analysis results are highly robust and are not significantly influenced by any individual study.
Table 5. Sensitivity analyses for long-term prognosis prediction.
Table 6. Sensitivity analyses for reperfusion prediction.
4 Discussion
The increasing popularity of radiomics prediction for assessing AIS patient outcomes is evident from the growing number of studies in recent years. This systematic review and meta-analysis identified 16 studies employing radiomic techniques to predict reperfusion and prognosis in AIS patients. Remarkably, radiomics model demonstrated promising performance, with pooled AUC values of 0.883 for long-term prognosis and 0.800 for reperfusion prediction, validating its potential value in AIS management.
Regarding the quality of studies in our meta-analysis, the average RQS was 14, representing 39.41% of the total RQS, with a median score of 13, corresponding to 36.11% of the total RQS. Most studies scored between 30 to 40%, while two studies scored below 30%, with one as low as 5.56%. Upon comparison with radiomics meta-analyses in other fields, we observed that the quality of radiomics studies is a widely concern. For instance, a review of cholangiocarcinoma encompassing 38 original studies reported a median RQS of 9, amounting to just 25.0% of the total RQS (52). Similarly, a review focusing on ovarian imaging, which included 63 studies, found a median RQS of 6, corresponding to 30.6% of the total RQS, indicating lower scoring (53). Another meta-analysis covering 57 ovarian cancer studies reported an average RQS of 30.7%, which is also considered unsatisfactory (54). Reviews in other areas of radiomics, such as prostate cancer (55), meningiomas (56), nasopharyngeal carcinoma (57), and cardiovascular fields (58), have also highlighted concerns regarding the quality of machine learning and radiomics research, underscoring the general insufficiency in the scientific rigor of radiomics studies (59). This draws attention to the need for a more standardized approach in the radiomics workflow before its integration into clinical practice, to enhance the quality and reliability of findings, thereby ensuring their utility and applicability in clinical settings.
Incorporating clinical characteristics into radiomics models in most cases highlights the importance of a comprehensive approach. Integrating a patient's clinical and radiomic profile can improve predictive accuracy. In fact, several studies demonstrated enhanced performance of prediction after integration, highlighting the advantages of using multidimensional information.
These findings emphasize the increasing acceptance and potential of radiomics in enhancing clinical decision-making for AIS patients. When these high-dimensional data are combined with machine learning models, they have the potential to optimize personalized treatment approaches. Such objective and time-sensitive risk stratification strategy can help with treatment decisions and enable tele-stroke assessment of patients. Particularly in the absence of reliable clinical information at the time of admission, models solely using radiomics features can serve as a valuable prognostication tool.
4.1 Limitation
A principal limitation concerns the quality assessment of included studies. The RQS tool is widely used for evaluating the quality of radiomics research. However, a recent study has highlighted the challenges in correctly understanding and implementing RQS to allocate reproducible scores (60), while inappropriate methodological quality assessment can lead to bias in quality assessments. Indeed, our study encountered this inter-rater reliability issue as well, particularly in the items of calibration statistics, validation, and open science and data, where our two evaluators assigned inconsistent scores for 6 included studies (Supplementary Table 5 and Supplementary Figure 1). We analyzed that these inconsistencies arose from the different expertise of the two raters in medical statistics and machine learning, affecting calibration statistics and open science and data, and varied academic experiences in radiology impacting validation item. Our current approach to address these inconsistencies is to average the divergent scores. However, we recognize that this simplistic method is a compromise and reflects a broader challenge faced by researchers using RQS tool. A robust and reproducible scoring tool for radiomics research is crucial for our study and for the field of radiomics. We eagerly anticipate the development of more standardized tools for evaluating radiomics research, which would not only aid our meta-analysis but also guide future studies in radiomics field.
Another critical limitation is evident in the current landscape of AIS radiomics studies. Firstly, for when predicting reperfusion, we observe significant heterogeneity among studies, which could not solely explained by single-center vs. multicenter design. Variations in study populations, methodologies, and the choice of utilizing first-pass effect recirculation results or multiple recirculation results for predicting MTB also could contribute to the inter-study heterogeneity. Indeed, previous studies have emphasized that radiomics studies are heterogeneous in various areas (52–55, 59, 61, 62). Subgroup analysis may partially address this heterogeneity. However, a comprehensive assessment of these factors will require more published studies. Moreover, due to the different parameters used in each article, we extracted AUC for the meta-analysis, as it was reported in all 16 studies. However, for a more comprehensive analysis, parameters such as model accuracy and precision need to be evaluated. Regrettably, due to constraints imposed by the original studies, we were unable to conduct a comprehensive evaluation using these additional parameters.
Additionally, in studies featuring multiple radiomics models, we selected the best-performing model for each. This approach, while enhancing comparability across studies, introducing a selection bias, potentially leading to overly optimistic estimates. We chose the best-performing models based on their ability to provide consistent and comparable benchmarks across studies. However, this necessitates a cautious interpretation of our findings, recognizing that it may limit the overall generalizability of the results. Future research would benefit from incorporating a broader range of models to provide a deeper understanding of predictive capabilities in the field.
Moreover, the dominance of retrospective studies raises concerns about selection bias and potentially overestimate of model performance. The lack of consideration for inter-scanner differences or vendor-dependent features further impact image quality and feature extraction. To enhance consistency and reproducibility, it is crucial to establish a unified protocol for image acquisition and processing. Moving forward, future studies should prioritize prospective, multicentric collaborations to validate and generalize the predictive power of radiomics models across diverse populations and clinical settings.
5 Conclusions
This review provides a quality assessment and meta-analysis of studies that using radiomics features to predict outcome in AIS patients, highlighting the potential of radiomics-based predictive models. As the field continues to grow, integrating radiomics-based machine learning models into clinical pathways is expected to improve personalized care of AIS patients. Future efforts should further leverage this potential to optimize patient outcomes.
Data availability statement
The original contributions presented in the study are included in the article/Supplementary material, further inquiries can be directed to the corresponding author.
Author contributions
JK: Conceptualization, Investigation, Methodology, Project administration, Software, Supervision, Validation, Writing – original draft, Writing – review & editing. DZ: Data curation, Formal analysis, Software, Validation, Visualization, Writing – review & editing.
Funding
The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fneur.2023.1335851/full#supplementary-material
References
2. Collaborators GBDCoD. Global, regional, and national age-sex specific mortality for 264 causes of death, 1980-2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet. (2017) 390:1151–210. doi: 10.1016/S0140-6736(17)32152-9
3. Wang W, Jiang B, Sun H, Ru X, Sun D, Wang L, et al. Prevalence, incidence, and mortality of stroke in China: results from a nationwide population-based survey of 480 687 adults. Circulation. (2017) 135:759–71. doi: 10.1161/CIRCULATIONAHA.116.025250
4. Wang YJ Li ZX, Gu HQ, Zhai Y, Jiang Y, Zhao XQ, et al. China stroke statistics 2019: a report from the national center for healthcare quality management in neurological diseases, China National Clinical Research Center for Neurological Diseases, the Chinese Stroke Association, National Center for Chronic and Non-communicable Disease Control and Prevention, Chinese Center for Disease Control and Prevention and Institute for Global Neuroscience and Stroke Collaborations. Stroke Vasc Neurol. (2020) 5:211–39. doi: 10.1136/svn-2020-000457
5. Liu M, Yan M, Guo Y, Xie Z, Li R, Li J, et al. Acute ischemic stroke at high altitudes in china: early onset and severe manifestations. Cells. (2021) 10:809. doi: 10.3390/cells10040809
6. Liu L, Chen W, Zhou H, Duan W, Li S, Huo X, et al. Chinese Stroke Association guidelines for clinical management of cerebrovascular disorders: executive summary and 2019 update of clinical management of ischaemic cerebrovascular diseases. Stroke Vasc Neurol. (2020) 5:159–76. doi: 10.1136/svn-2020-000378
7. Mendelson SJ, Prabhakaran S. Diagnosis and management of transient ischemic attack and acute ischemic stroke: a review. JAMA. (2021) 325:1088–98. doi: 10.1001/jama.2020.26867
8. Hurford R, Sekhar A, Hughes TAT, Muir KW. Diagnosis and management of acute ischaemic stroke. Pract Neurol. (2020) 20:304–16. doi: 10.1136/practneurol-2020-002557
9. Yi Y, Liu Z, Wang M, Sun M, Jiang X, Ma C, et al. Penumbra in acute ischemic stroke. Curr Neurovasc Res. (2021) 18:572–85. doi: 10.2174/1567202619666211231094046
10. Hung MC, Hsieh CL, Hwang JS, Jeng JS, Wang JD. Estimation of the long-term care needs of stroke patients by integrating functional disability and survival. PLoS ONE. (2013) 8:e75605. doi: 10.1371/journal.pone.0075605
11. Mayerhoefer ME, Materka A, Langs G, Haggstrom I, Szczypinski P, Gibbs P, et al. Introduction to radiomics. J Nucl Med. (2020) 61:488–95. doi: 10.2967/jnumed.118.222893
12. Puig J, Shankar J, Liebeskind D, Terceno M, Nael K, Demchuk AM, et al. From “Time is Brain” to “Imaging is Brain”: a paradigm shift in the management of acute ischemic stroke. J Neuroimaging. (2020) 30:562–71. doi: 10.1111/jon.12693
13. Quan G, Ban R, Ren JL, Liu Y, Wang W, Dai S, et al. FLAIR and ADC image-based radiomics features as predictive biomarkers of unfavorable outcome in patients with acute ischemic stroke. Front Neurosci. (2021) 15:730879. doi: 10.3389/fnins.2021.730879
14. Vaclavik D, Volny O, Cimflova P, Svub K, Dvornikova K, Bar M. The importance of CT perfusion for diagnosis and treatment of ischemic stroke in anterior circulation. J Integr Neurosci. (2022) 21:92. doi: 10.31083/j.jin2103092
15. Joundi RA, Menon BK. Thrombus composition, imaging, and outcome prediction in acute ischemic stroke. Neurology. (2021) 97(Suppl. 2):S68–78. doi: 10.1212/WNL.0000000000012796
16. Naik A, Adeleye O, Koester SW, Winkler EA, Hartke JN, Karahalios K, et al. Cerebrospinal fluid biomarkers for diagnosis and the prognostication of acute ischemic stroke: a systematic review. Int J Mol Sci. (2023) 24:10902. doi: 10.3390/ijms241310902
17. Nicolas-Jilwan M, Wintermark M. Automated brain perfusion imaging in acute ischemic stroke: interpretation pearls and pitfalls. Stroke. (2021) 52:3728–38. doi: 10.1161/STROKEAHA.121.035049
18. Shatte ABR, Hutchinson DM, Teague SJ. Machine learning in mental health: a scoping review of methods and applications. Psychol Med. (2019) 49:1426–48. doi: 10.1017/S0033291719000151
19. Li B, Feridooni T, Cuen-Ojeda C, Kishibe T, de Mestral C, Mamdani M, et al. Machine learning in vascular surgery: a systematic review and critical appraisal. NPJ Digit Med. (2022) 5:7. doi: 10.1038/s41746-021-00552-y
20. Windisch P, Koechli C, Rogers S, Schroder C, Forster R, Zwahlen DR, et al. Machine learning for the detection and segmentation of benign tumors of the central nervous system: a systematic review. Cancers. (2022) 14:2676. doi: 10.3390/cancers14112676
21. Jiang F, Jiang Y, Zhi H, Dong Y, Li H, Ma S, et al. Artificial intelligence in healthcare: past, present and future. Stroke Vasc Neurol. (2017) 2:230–43. doi: 10.1136/svn-2017-000101
22. Heo J, Yoon JG, Park H, Kim YD, Nam HS, Heo JH. Machine learning-based model for prediction of outcomes in acute stroke. Stroke. (2019) 50:1263–5. doi: 10.1161/STROKEAHA.118.024293
23. Sheth SA, Giancardo L, Colasurdo M, Srinivasan VM, Niktabe A, Kan P. Machine learning and acute stroke imaging. J Neurointerv Surg. (2023) 15:195–9. doi: 10.1136/neurintsurg-2021-018142
24. Powers WJ, Rabinstein AA, Ackerson T, Adeoye OM, Bambakidis NC, Becker K, et al. 2018 guidelines for the early management of patients with acute ischemic stroke: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke. (2018) 49:e46–e110. doi: 10.1161/STR.0000000000000158
25. Powers WJ, Rabinstein AA, Ackerson T, Adeoye OM, Bambakidis NC, Becker K, et al. Guidelines for the early management of patients with acute ischemic stroke: 2019 update to the 2018 guidelines for the early management of acute ischemic stroke: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke. (2019) 50:e344–418. doi: 10.1161/STR.0000000000000211
26. Jauch EC, Saver JL, Adams HP Jr., Bruno A, Connors JJ, Demaerschalk BM, et al. Guidelines for the early management of patients with acute ischemic stroke: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke. (2013) 44:870–947. doi: 10.1161/STR.0b013e318284056a
27. Hacke W, Albers G, Al-Rawi Y, Bogousslavsky J, Davalos A, Eliasziw M, et al. The Desmoteplase in Acute Ischemic Stroke Trial (DIAS): a phase II MRI-based 9-hour window acute stroke thrombolysis trial with intravenous desmoteplase. Stroke. (2005) 36:66–73. doi: 10.1161/01.STR.0000149938.08731.2c
28. Albers GW, Thijs VN, Wechsler L, Kemp S, Schlaug G, Skalabrin E, et al. Magnetic resonance imaging profiles predict clinical response to early reperfusion: the diffusion and perfusion imaging evaluation for understanding stroke evolution (DEFUSE) study. Ann Neurol. (2006) 60:508–17. doi: 10.1002/ana.20976
29. Hacke W, Furlan AJ, Al-Rawi Y, Davalos A, Fiebach JB, Gruber F, et al. Intravenous desmoteplase in patients with acute ischaemic stroke selected by MRI perfusion-diffusion weighted imaging or perfusion CT (DIAS-2): a prospective, randomised, double-blind, placebo-controlled study. Lancet Neurol. (2009) 8:141–50. doi: 10.1016/S1474-4422(08)70267-9
30. Chen Y, He Y, Jiang Z, Xie Y, Nie S. Ischemic stroke subtyping method combining convolutional neural network and radiomics. J Xray Sci Technol. (2023) 31:223–35. doi: 10.3233/XST-221284
31. Zhang R, Zhu L, Zhu Z, Ge Y, Zhang Z, Wang T. Apparent diffusion coefficient map based radiomics model in identifying the ischemic penumbra in acute ischemic stroke. Ann Palliat Med. (2020) 9:2684–92. doi: 10.21037/apm-20-1142
32. Sollini M, Antunovic L, Chiti A, Kirienko M. Towards clinical application of image mining: a systematic review on artificial intelligence and radiomics. Eur J Nucl Med Mol Imaging. (2019) 46:2656–72. doi: 10.1007/s00259-019-04372-x
33. Lambin P, Leijenaar RTH, Deist TM, Peerlings J, de Jong EEC, van Timmeren J, et al. Radiomics: the bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol. (2017) 14:749–62. doi: 10.1038/nrclinonc.2017.141
34. Moons KGM, Wolff RF, Riley RD, Whiting PF, Westwood M, Collins GS, et al. PROBAST: a tool to assess risk of bias and applicability of prediction model studies: explanation and elaboration. Ann Intern Med. (2019) 170:W1–W33. doi: 10.7326/M18-1377
35. Debray TP, Damen JA, Riley RD, Snell K, Reitsma JB, Hooft L, et al. A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes. Stat Methods Med Res. (2019) 28:2768–86. doi: 10.1177/0962280218785504
36. Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. (2003) 327:557–60. doi: 10.1136/bmj.327.7414.557
37. Hofmeister J, Bernava G, Rosi A, Vargas MI, Carrera E, Montet X, et al. Clot-based radiomics predict a mechanical thrombectomy strategy for successful recanalization in acute ischemic stroke. Stroke. (2020) 51:2488–94. doi: 10.1161/STROKEAHA.120.030334
38. Zhang L, Wu J, Yu R, Xu R, Yang J, Fan Q, et al. Non-contrast CT radiomics and machine learning for outcomes prediction of patients with acute ischemic stroke receiving conventional treatment. Eur J Radiol. (2023) 165:110959. doi: 10.1016/j.ejrad.2023.110959
39. Li L, Li M, Chen Z, Lu F, Zhao M, Zhang H, et al. Prognostic value of radiomics-based hyperdense middle cerebral artery sign for patients with acute ischemic stroke after thrombectomy strategy. Front Neurol. (2022) 13:1037204. doi: 10.3389/fneur.2022.1037204
40. Patel TR, Santo BA, Baig AA, Waqas M, Monterio A, Levy EI, et al. Histologically interpretable clot radiomic features predict treatment outcomes of mechanical thrombectomy for ischemic stroke. Neuroradiology. (2023) 65:737–49. doi: 10.1007/s00234-022-03109-2
41. Xiong X, Wang J, Ke J, Hong R, Jiang S, Ye J, et al. Radiomics-based intracranial thrombus features on preoperative noncontrast CT predicts successful recanalization of mechanical thrombectomy in acute ischemic stroke. Quant Imaging Med Surg. (2023) 13:682–94. doi: 10.21037/qims-22-599
42. Ramos LA, van Os H, Hilbert A, Olabarriaga SD, van der Lugt A, Roos Y, et al. Combination of radiological and clinical baseline data for outcome prediction of patients with an acute ischemic stroke. Front Neurol. (2022) 13:809343. doi: 10.3389/fneur.2022.809343
43. Qiu W, Kuang H, Nair J, Assis Z, Najm M, McDougall C, et al. Radiomics-based intracranial thrombus features on CT and CTA predict recanalization with intravenous alteplase in patients with acute ischemic stroke. AJNR Am J Neuroradiol. (2019) 40:39–44. doi: 10.3174/ajnr.A5918
44. Tang TY, Jiao Y, Cui Y, Zhao DL, Zhang Y, Wang Z, et al. Penumbra-based radiomics signature as prognostic biomarkers for thrombolysis of acute ischemic stroke patients: a multicenter cohort study. J Neurol. (2020) 267:1454–63. doi: 10.1007/s00415-020-09713-7
45. Cui H, Wang X, Bian Y, Song S, Feng DD. Ischemic stroke clinical outcome prediction based on image signature selection from multimodality data. Annu Int Conf IEEE Eng Med Biol Soc. (2018) 2018:722–5. doi: 10.1109/EMBC.2018.8512291
46. Tolhuisen ML, Hoving JW, Koopman MS, Kappelhof M, van Voorst H, Bruggeman AE, et al. Outcome prediction based on automatically extracted infarct core image features in patients with acute ischemic stroke. Diagnostics. (2022) 12:1786. doi: 10.3390/diagnostics12081786
47. Ye W, Chen X, Li P, Tao Y, Wang Z, Gao C, et al. OEDL: an optimized ensemble deep learning method for the prediction of acute ischemic stroke prognoses using union features. Front Neurol. (2023) 14:1158555. doi: 10.3389/fneur.2023.1158555
48. Avery EW, Behland J, Mak A, Haider SP, Zeevi T, Sanelli PC, et al. CT angiographic radiomics signature for risk stratification in anterior large vessel occlusion stroke. Neuroimage Clin. (2022) 34:103034. doi: 10.1016/j.nicl.2022.103034
49. Jiang L, Miao Z, Chen H, Geng W, Yong W, Chen YC, et al. Radiomics analysis of diffusion-weighted imaging and long-term unfavorable outcomes risk for acute stroke. Stroke. (2023) 54:488–98. doi: 10.1161/STROKEAHA.122.040418
50. Yu H, Wang Z, Sun Y, Bo W, Duan K, Song C, et al. Prognosis of ischemic stroke predicted by machine learning based on multi-modal MRI radiomics. Front Psychiatry. (2022) 13:1105496. doi: 10.3389/fpsyt.2022.1105496
51. Zhang H, Polson J, Nael K, Salamon N, Yoo B, Speier W, et al. A machine learning approach to predict acute ischemic stroke thrombectomy reperfusion using discriminative MR image features. IEEE EMBS Int Conf Biomed Health Inform. (2021) 2021. doi: 10.1109/BHI50953.2021.9508597
52. Cannella R, Vernuccio F, Klontzas ME, Ponsiglione A, Petrash E, Ugga L, et al. Systematic review with radiomics quality score of cholangiocarcinoma: an EuSoMII Radiomics Auditing Group Initiative. Insights Imag. (2023) 14:21. doi: 10.1186/s13244-023-01365-1
53. Ponsiglione A, Stanzione A, Spadarella G, Baran A, Cappellini LA, Lipman KG, et al. Ovarian imaging radiomics quality score assessment: an EuSoMII radiomics auditing group initiative. Eur Radiol. (2023) 33:2239–47. doi: 10.1007/s00330-022-09180-w
54. Huang ML, Ren J, Jin ZY, Liu XY, He YL Li Y, et al. A systematic review and meta-analysis of CT and MRI radiomics in ovarian cancer: methodological issues and clinical utility. Insights Imag. (2023) 14:117. doi: 10.1186/s13244-023-01464-z
55. Stanzione A, Gambardella M, Cuocolo R, Ponsiglione A, Romeo V, Imbriaco M. Prostate MRI radiomics: a systematic review and radiomic quality score assessment. Eur J Radiol. (2020) 129:109095. doi: 10.1016/j.ejrad.2020.109095
56. Ugga L, Perillo T, Cuocolo R, Stanzione A, Romeo V, Green R, et al. Meningioma MRI radiomics and machine learning: systematic review, quality score assessment, and meta-analysis. Neuroradiology. (2021) 63:1293–304. doi: 10.1007/s00234-021-02668-0
57. Spadarella G, Calareso G, Garanzini E, Ugga L, Cuocolo A, Cuocolo R, et al. based radiomics in nasopharyngeal cancer: Systematic review and perspectives using radiomic quality score (RQS) assessment. Eur J Radiol. (2021) 140:109744. doi: 10.1016/j.ejrad.2021.109744
58. Ponsiglione A, Stanzione A, Cuocolo R, Ascione R, Gambardella M, De Giorgi M, et al. Cardiac CT and MRI radiomics: systematic review of the literature and radiomics quality score assessment. Eur Radiol. (2022) 32:2629–38. doi: 10.1007/s00330-021-08375-x
59. Park JE, Kim D, Kim HS, Park SY, Kim JY, Cho SJ, et al. Quality of science and reporting of radiomics in oncologic studies: room for improvement according to radiomics quality score and TRIPOD statement. Eur Radiol. (2020) 30:523–36. doi: 10.1007/s00330-019-06360-z
60. Akinci D'Antonoli T, Cavallo AU, Vernuccio F, Stanzione A, Klontzas ME, Cannella R, et al. Reproducibility of radiomics quality score: an intra- and inter-rater reliability study. Eur Radiol. (2023). doi: 10.1007/s00330-023-10217-x. [Epub ahead of print].
61. Ursprung S, Beer L, Bruining A, Woitek R, Stewart GD, Gallagher FA, et al. Radiomics of computed tomography and magnetic resonance imaging in renal cell carcinoma-a systematic review and meta-analysis. Eur Radiol. (2020) 30:3558–66. doi: 10.1007/s00330-020-06666-3
Keywords: acute ischemic stroke, radiomics, prognosis prediction, reperfusion prediction, systematic review and meta-analysis
Citation: Kong J and Zhang D (2024) Current status and quality of radiomics studies for predicting outcome in acute ischemic stroke patients: a systematic review and meta-analysis. Front. Neurol. 14:1335851. doi: 10.3389/fneur.2023.1335851
Received: 09 November 2023; Accepted: 15 December 2023;
Published: 02 January 2024.
Edited by:
Francesco Santini, University of Basel, SwitzerlandReviewed by:
Thomas Lindner, University of Hamburg, GermanyMichail Klontzas, University Hospital of Heraklion, Greece
Copyright © 2024 Kong and Zhang. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Jinfen Kong, kongjinfen429@163.com