Shraddha Mainali
Marin E. Darsie
Keaton S. Smetana- 1Department of Neurology, Virginia Commonwealth University, Richmond, VA, United States
- 2Department of Emergency Medicine, University of Wisconsin Hospitals and Clinics, Madison, WI, United States
- 3Department of Neurological Surgery, University of Wisconsin Hospitals and Clinics, Madison, WI, United States
- 4Department of Pharmacy, The Ohio State University Wexner Medical Center, Columbus, OH, United States
The application of machine learning has rapidly evolved in medicine over the past decade. In stroke, commercially available machine learning algorithms have already been incorporated into clinical application for rapid diagnosis. The creation and advancement of deep learning techniques have greatly improved clinical utilization of machine learning tools and new algorithms continue to emerge with improved accuracy in stroke diagnosis and outcome prediction. Although imaging-based feature recognition and segmentation have significantly facilitated rapid stroke diagnosis and triaging, stroke prognostication is dependent on a multitude of patient specific as well as clinical factors and hence accurate outcome prediction remains challenging. Despite its vital role in stroke diagnosis and prognostication, it is important to recognize that machine learning output is only as good as the input data and the appropriateness of algorithm applied to any specific data set. Additionally, many studies on machine learning tend to be limited by small sample size and hence concerted efforts to collate data could improve evaluation of future machine learning tools in stroke. In the present state, machine learning technology serves as a helpful and efficient tool for rapid clinical decision making while oversight from clinical experts is still required to address specific aspects not accounted for in an automated algorithm. This article provides an overview of machine learning technology and a tabulated review of pertinent machine learning studies related to stroke diagnosis and outcome prediction.
Introduction
The term machine learning (ML) was coined by Arthur Samuel in 1959 (1). He investigated two machine learning procedures using the game of checkers and concluded that computers can be programmed quickly to play a better game of checkers than the person who wrote the program. Simply put, machine learning can be defined as a subfield of artificial intelligence (AI) that uses computerized algorithms to automatically improve performance through iterative learning process or experience (i.e., data acquisition) (2). Of late, the field of ML has vastly evolved with the development of various computerized algorithms for pattern recognition and data assimilation to improve predictions, decisions, perceptions, and actions across various fields and serves as an extension to the traditional statistical approaches. In our day-to-day life, a relatable example of ML is the application of spam filters to the 319 billion emails sent and received daily worldwide, of which, nearly 50% can be classified as spam (3). Use of ML technology has made this process efficient and manageable. The ML technology utilizes various methods for automated data analysis including linear and logistic regression models as well as other methods such as the support vector machines (SVM), random forests (RF), classification trees and discriminant analysis that allow combination of features (data points) in a non-linear manner with flexible decision boundaries. The advent of neural networks and deep learning (DL) technology has transformed the field of ML with automatic and efficient feature identification and processing within a covert analytic network, without the need for a priori feature selection. Notably, performance of DL is known to improve with access to larger datasets, whereas classic ML methods tend to plateau at relatively lower performance levels. Hence, in this era of big data where clinicians are constantly inundated with plethora of clinical information, use of DL technology has significnalty enhanced our ability to assimilate the vast amount of clinical data to make expeditious clinical decision.
Stroke is a leading cause of death, disability, and cognitive impairment in the United States (4). According to the 2013 policy statement from the American Heart Association, an estimated 4% of US adults will suffer from a stroke by 2030, accounting for total annual stroke-related medical cost of $240.67 billion by 2030 (5). For ischemic stroke, acute management is highly dependent on prompt diagnosis. According to the current ischemic stroke guidelines, patients are eligible for intravenous thrombolysis up to 4.5 h from symptom onset and endovascular thrombectomy without advanced imaging within 6 h of symptom onset (6–8). For patients presenting between 6 and 24 h of symptom onset (or last known well time), advanced imaging is recommended to assess salvageable penumbra for decisions regarding endovascular therapy (9–11). Similarly for hemorrhagic stroke, timely diagnosis utilizing imaging technology to evaluate the type and etiology of hemorrhage is important in guiding acute treatment decisions. Prompt diagnosis with emergent treatment decision and accurate prognostication is hence the cornerstone of acute stroke management. Over the recent years, a multitude of ML methodologies have been applied to stroke for various purposes, including diagnosis of stroke (12, 13), prediction of stroke symptom onset (14, 15), assessment of stroke severity (16, 17), characterization of clot composition (18), analysis of cerebral edema (19), prediction of hematoma expansion (20), and outcome prediction (21–23). In particular, there has been a rapid increase in the trend of ML application for imaging-based stroke diagnosis and outcome prediction. The Ischemic Stroke Lesion Segmentation Challenge (ISLES: http://www.isles-challenge.org/) provides a global competing platform encouraging teams across the world to develop advanced tools for stroke lesion analysis using ML. In this platform, competitors train their algorithms on a standardized dataset and eventually generate benchmarks for algorithm performance.
Deciding which type of ML to use on a specific dataset depends on factors such as the size of dataset, need for supervision, ability to learn, and the generalizability of the model (24). DL technology such as the deep neural networks has significantly improved the ability for image segmentation, automated featurization (e.g., conversion of raw signal into clinically useful parameter), and multimodal prognostication in stroke; and it is increasingly utilized in stroke-based applications (25–27). For example, DL algorithms can be applied to extract meaningful imaging features for image processing in an increasing order of hierarchical complexity to make predictions, such as the final infarct volume (27). Some commonly used ML types with their respective algorithms and practical examples are outlined in Figures 1–3. In the healthcare setting, supervised and unsupervised algorithms are both commonly used. In this review, we will specifically focus on ML strategies for stroke diagnosis and outcome prediction. Table 1 provides an overview of pertinent studies with use of ML in stroke diagnosis (Section A) and outcome prediction (Section B). A glossary of machine learning terms with brief description is separately provided in Supplementary Table 1.
Figure 1. Supervised learning. In supervised learning, a model is built by labeling images [Subarachnoid Hemorrhage (SAH) and Not Subarachnoid Hemorrhage (Not SAH)], a predictive model is created, and then tested for accuracy in reading unlabeled images (gray box). Source: WesternDigital BLOG.
Figure 2. Unsupervised learning. In unsupervised learning, the machine learning algorithm discovers structures within given data. The initial data is not labeled and a clustering algorithm groups unlabeled data together. Source: WesternDigital BLOG.
Table 1. Studies utilizing machine learning for stroke diagnosis and prediction.
Methods
We searched PubMed, Google Scholar, Web of Science, and IEEE Xplore® for relevant articles using various combination of the following key words: “machine learning,” “artificial intelligence,” “stroke,” “ischemic stroke,” “hemorrhagic stroke,” “diagnosis,” “prognosis,” “outcome,” “big data,” and “outcome prediction.” Resulting abstracts were screened by all authors and articles were hand-picked for full review based on relevance and scientific integrity. Final article list was reviewed and approved by all authors.
Machine Learning in Stroke Diagnosis
The time-sensitive nature of stroke care underpins the need for accurate and rapid tools to assist in stroke diagnosis. Over the recent years, the science of brain imaging has vastly advanced with the availability of a myriad of AI based diagnostic imaging algorithms (77). Machine learning is particularly useful in diagnosis of acute stroke with large vessel occlusion (LVO). Various automated methods for detection of stroke core and penumbra size as well as mismatch quantification and detection of vascular thrombi have recently been developed (77). Over the past decade, 13 different companies have developed automated and semi-automated commercially available software for acute stroke diagnostics (Aidoc®, Apollo Medical Imaging Technology®, Brainomix®, inferVISION®, RAPID®, JLK Inspection®, Max-Q AI®, Nico.lab®, Olea Medical®, Qure.ai®, Viz.ai®, and Zebra Medical Vision®) (78). The RapidAI® and Viz.ai® technology have been approved under the medical device category of computer-assisted triage by the United States Food and Drug Administration (FDA). The RAPID MRI® (Rapid processing of Perfusion and Diffusion) software allows for an unsupervised, fully-automated processing of perfusion and diffusion data to identify those who may benefit from thrombectomy based on the mismatch ratio (79). Such commercial platforms available for automatic detection of ischemic stroke and LVO have facilitated rapid treatment decisions. When compared to manual segmentation of lesion volume and mismatch identification from patients enrolled in DEFUSE 2, the RAPID results were found to be well-correlated (r2 = 0.99 and 0.96 for diffusion and perfusion weighted imaging, respectively) with 100% sensitivity and 91% specificity for mismatch identification (80). Since 2008, the RapidAI® platform has expanded to include other products (Rapid® ICH, ASPECTS, CTA, LVO, CTP, MRI, Angio, and Aneurysm) that assist across the entire spectrum of stroke. Viz LVO® was the first FDA-cleared software to detect and alert clinicians of LVO via the “Viz Platform” (81). In a recent single center study with 1,167 CTAs analyzed, Viz LVO® was found to have a sensitivity of 0.81 and a negative predictive value of 0.99 with an accuracy of 0.94 (82).
Other areas of stroke diagnostics that have seen an increase in attention over the past decade are the identification of intracerebral hemorrhage (ICH) and patients at risk for delayed cerebral ischemia in the setting of aneurysmal subarachnoid hemorrhage (aSAH). While most studies tend to have good accuracy in detecting an ICH there is more variability in subclassification and measurements of hematoma volume. A summary of recent publications on ML in stroke diagnosis is presented in Table 1 (Section A).
Machine Learning in Stroke Outcome Prediction
Despite recent advances in stroke care, it remains the second leading cause of death and disability world-wide (4, 83). Although acute stroke diagnosis and determination of the time of stroke onset are the initial steps of comprehensive stroke management, clinicians are also often charged with the task of determining stroke outcomes. These outcomes range from discrete radiological outcomes (e.g., final infarct volume, the likelihood of hemorrhagic transformation, etc.), the likelihood of morbidity (e.g., stroke-associated pneumonia) and mortality, and various measures of functional independence (e.g., mRS score, Barthel Index score, cognitive, and language function, etc.).
Prognostication after an acute brain injury is notoriously challenging, particularly within the first 24–48 h (84). However, a clinician may be called upon to provide estimates of a patient's short-term and long-term mortality and degree of functional dependence to assist with decision-making regarding the intensity of care (e.g., use of thrombolytics or endovascular treatment, intubation, code status, etc.) (60, 64, 66, 67, 69, 70, 72–76). Like all medical emergencies, it is incumbent upon the stroke clinician to ensure that all care provided is concordant with an individual patient's goals (85). For example, a surrogate decision-maker may decline to reverse a patient's longstanding “do not intubate” order to facilitate mechanical thrombectomy if the clinician predicts the patient has a high likelihood of functional dependence or short-term mortality. Hence, accuracy in outcome prediction is critical in guiding management of our patients.
Determining a patient's likelihood of developing symptomatic intracranial hemorrhage (sICH) is of obvious, immediate value in acute stroke management in determining candidacy for thrombolytic therapy or endovascular treatment. Historically, clinician-based prognostication tools to predict the risk of symptomatic intracranial hemorrhage after IV thrombolysis, such as the SEDAN (Sugar, Early Infarct signs, Dense cerebral artery sign, Age, and NIHSS) and HAT (Hemorrhage After Thrombolysis) scores have been used to predict the risk of symptomatic intracranial hemorrhage after IV thrombolysis (23). Advances in ML and DL have allowed for the development of more accurate models which outperform the traditional SEDAN and HAT scores (23, 54, 55). Similarly, the ability to predict final infarct volume and the likelihood of the development of malignant cerebral edema have important treatment implications and remain a significant focus of ML in stroke (26, 51–53).
In patients with intracerebral hemorrhage (ICH), the ICH-score is one of the most widely used clinical prediction scores (85–88). Although ML technology for outcome prediction has rapidly advanced for ischemic stroke, recent ML studies predicting functional outcomes after ICH have also demonstrated high-discriminating power (63, 89). A recent study by Sennfält et al. tracked long-term functional dependence and mortality after an acute ischemic stroke of more than 20,000 Swedish patients (90). The 30-day mortality rate was 11.1%. At 5 years, 70.6% of ischemic stroke patients were functionally dependent (defined as mRS score of ≥3) or had died (5-year mortality rate of 50.6%). These sobering outcomes partially account for the development of many stroke prognostic models over the years, which frequently serve as benchmarks in stroke research. Recently, Matsumoto et al. compared the performance of six existing stroke prognostic models for predicting poor functional outcomes and in-hospital mortality with linear regression or decision tree ensemble models (59). The novel prediction models performed slightly better than the conventional models in predicting poor functional outcomes (AUC 0.88–0.94 vs. AUC 0.70–0.92) but were equivalent or marginally worse in predicting in-hospital death (AUC 0.84–0.88 vs. AUC 0.87–0.88). Many such stroke prediction models have emerged over the recent years. An overview of ML based automated algorithms for stroke outcome prediction is provided in Table 1 (Section B).
Discussion
In recent years, some DL algorithms have approached human levels of performance in object recognition (91). One of the greatest strengths of ML is its ability to endlessly process data and tirelessly perform an iterative task. Further, creation of a ML model can be performed much faster (i.e., in a matter of 5–6 days compared with 5–6 months or even years) than traditional computer-aided detection and diagnosis (CAD) (92). which makes ML an attractive field for computer experts and scientists. Several ML tools are currently in use including the FDA-approved ML algorithms previously discussed for rapid stroke diagnosis which have significantly enhanced the workflow of acute ischemic stroke patients.
Despite the prolific advent of new and improved ML algorithms with increasing clinical applications, it is important to recognize that computer-based algorithms are only as good as the data used to train the models. For a reliable algorithm, it is important to develop well-defined training, validation, and testing sets. Testing should be done on a diverse set of data points reflective of a real-world scenario. Overfitting can be an issue in ML algorithms when the model is trained on a group of highly-selected, specific features, which when tested on a larger dataset with varied features, fails to perform adequately. Similarly, underfitting can occur when a model is oversimplified with generalized feature selection in the training set which then becomes unable to capture the relevant features within a complex pattern of a larger or more diverse testing set. The aphorism “garbage in, garbage out” remains true as the use of inadequate or unvalidated data points (e.g., unverified clinical reports from electronic health record) in the training set can lead to poor performance of the ML algorithm in the testing set. Hence, it is important to note that the algorithmic decision-making tools do not guarantee accurate and unbiased interpretation compared to established logistic regression models (56, 59, 93). Comparisons to well-established models should be standard when developing new ML algorithms given the high cost associated with ML (e.g., the time required to collect data, train the model, perform internal and external validations, cost of reliable and secure data storage, etc.) (94). Specifically, as it relates to diagnostics there are a myriad of considerations that must be taken into account. Not only should the algorithm provide accurate information quickly, but it should have the ability to integrate into the electornic health record (EHR) to improve end user experience and efficiency in workflow. Programs such as RAPID®, Viz.ai®, and Brainomix® have started to successfully integrate into the EHR, which has helped expedite acute stroke diagnosis and triage process. One of the major technical challenges of ML include the ability to develop an algorithm with a “reasonable” detection rate of pathology without an excessive rate of false-positives. For example, there are notable discrepancies among various ML studies for ICH diagnosis, with varying accuracy depending on the type of ICH (e.g., spontaneous ICH, SDH, aSAH, or IVH). Overfitting and underfitting of the model could lead to poor applicability and therefore, image preprocessing with meticulous feature selection is necessary. Furthermore, the “black-box” nature of ML precludes the clinicians from identifying and addressing biases within the algorithms (95, 96). Hence, proper external validation is necessary to ensure generalizability of the algorithm in diverse clinical scenarios.
For stroke prediction, most existing ML algorithms utilize dichotomized outcomes. Functional outcome is frequently defined as “good” when mRS score is 0–2 and “poor” when mRS score is 3–6 by convention and IS studies often measure mRS score at 90 days after stroke (64–69, 97). However, the medical community is increasingly embracing patient-centered outcomes. People are starting to recognize the need for longitudinal patient follow-up given potential for functional improvement beyond conventional norms of 90 days (98). Once patient-centered outcomes are clinically validated (e.g., MRS cutoff of 0–2 vs. 3–6, 0–3 vs. 4–6, or 0–4 vs. 5–6), new ML algorithms incorporating such outcomes would be increasingly helpful to the clinicians. The use of high-yield, ML programs using patient-centered outcomes could ease the commonplace but challenging discussions of the anticipated quality of life and the risk of long-term dependency or death before deciding on a patient's goals-of-care. It is however important to apply caution while using ML algorithms for outcome prediction as patient demographics and clinical practice continue to evolve and updates to the ML algorithms would be necessary to remain applicable to evolving patient populations and clinical standards. Additionally, developers often retrieve data from existing datasets (e.g., clinical trial data) with its inherent biases including selection bias, observer bias and other confounders (e.g., withdrawal of life supporting therapy may be more common in older patients with large hemispheric stroke compared to younger patients, which could confound outcome prediction in older patients compared to younger ones).
Overall, compared to other diseases such as Alzheimer's disease, there is a relative paucity of large, high-quality datasets within stroke. Some limitations that have stymied the development of large, open-access stroke registries include the need for data-sharing agreements, patient privacy concerns, high costs of data storage and security, arbitration of quality control of the input data, etc. (95). Cohesive and collaborative efforts across hospital systems, regions, and nations with data acquisition and harmonization is needed to improve future ML-based programs in stroke. With adoption of EHR systems, healthcare data is rapidly accumulating with an estimated over 35 zettabytes of existing healthcare data! (99). Adoption of AI and ML algorithms allow us to efficiently process the plethora of information that surround us every day. Nonetheless, as we continue to adapt to this evolving landscape of medical practice surrounding big data, clinicians need to remain aware of the limitations of this modern day “black box” magic.
Conclusion
The emerging ML technology has rapidly integrated into multiple fields of medicine including stroke. Deep learning has significantly enhanced practical applications of ML and some newer algorithms are known to have comparable accuracy to humans. However, the diagnosis and prognosis of a disease, including stroke, is highly intricate and depends on various clinical and personal factors. The development of optimal ML programs requires comprehensive data collection and assimilation to improve diagnostic and prognostic accuracy. Given the “black box” or cryptic nature of these algorithms, it is extremely important for the end-user (i.e., clinicians) to understand the intended use and limitations of any ML algorithm to avoid inaccurate data interpretation. Although ML algorithms have improved stroke systems of care, blind dependence on such computerized technology may lead to misdiagnosis or inaccurate prediction of prognostic trajectories. At the current state, ML tools are best used as “aids” for clinical decision making while still requiring oversight to address relevant clinical aspects that are overlooked by the algorithm.
Author Contributions
SM: substantial contributions including conception and design of the work, literature review, interpretation and summarization of data, drafting the complete manuscript, revising it critically for important intellectual content, and final approval of the manuscript to be published. MD and KS: contribution including conception and design of the work, literature review, interpretation and summarization of the data, drafting of critical portion of the manuscript, critical revision for important intellectual content, and final approval of the manuscript. All authors contributed to the article and approved the submitted version.
Funding
This article was supported by the Virginia Commonwealth University, Department of Neurology.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher's Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary Material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fneur.2021.734345/full#supplementary-material
References
1. Samuel L. Some studies in machine learning using the game of checkers. IBM J Res Dev. (1959) 3:210–29. doi: 10.1147/rd.33.0210
3. Tankovska H. E-mail Usage in the United States—Statistics and Facts. In, Statista. Hamburg: Statista (2021).
4. GBD 2016 Neurology Collaborators. Global, regional, and national burden of neurological disorders:1990–2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet Neurol. (2019) 18:459–80. doi: 10.1016/S1474-4422(18)30499-X
5. Ovbiagele B, Goldstein LB, Higashida RT, Howard VJ, Johnston SC, Khavjou OA, et al. Forecasting the future of stroke in the United States: a policy statement from the American Heart Association and American Stroke Association. Stroke. (2013) 44:2361–75. doi: 10.1161/STR.0b013e31829734f2
6. Hacke W, Kaste M, Bluhmki E, Brozman M, Dávalos A, Guidetti D, et al. Thrombolysis with alteplase 3 to 45 hours after acute ischemic stroke. N Engl J Med. (2008) 359:1317–29. doi: 10.1056/NEJMoa0804656
7. Bluhmki E, Chamorro A, Dávalos A, Machnig T, Sauce C, Wahlgren N, et al. Stroke treatment with alteplase given 30-45 h after onset of acute ischaemic stroke (ECASS III): additional outcomes and subgroup analysis of a randomised controlled trial. Lancet Neurol. (2009) 8:1095–102. doi: 10.1016/S1474-4422(09)70264-9
8. Powers WJ, Rabinstein AA, Ackerson T, Adeoye OM, Bambakidis NC, Becker K, et al. 2018 Guidelines for the early management of patients with acute ischemic stroke: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke. (2018) 49:e46–110. doi: 10.1161/STR.0000000000000158
9. Nogueira RG, Jadhav AP, Haussen DC, Bonafe A, Budzik RF, Bhuva P, et al. Thrombectomy 6 to 24 hours after stroke with a mismatch between deficit and infarct. N Engl J Med. (2018) 378:11–21. doi: 10.1056/NEJMoa1706442
10. Campbell A, Bruce CV, Peter Mitchell J, Timothy Kleinig J, Helen Dewey M, Churilov L, et al. Endovascular therapy for ischemic stroke with perfusion-imaging selection. N Engl J Med. (2015) 372:1009–18. doi: 10.1056/NEJMoa1414792
11. Powers WJ, Rabinstein AA, Ackerson T, Adeoye OM, Bambakidis NC, Becker K, et al. Guidelines for the early management of patients with acute ischemic stroke: 2019 update to the 2018 guidelines for the early management of acute ischemic stroke: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke. (2019) 50:e344–418. doi: 10.1161/STR.0000000000000211
12. Ni Y, Alwell K, Moomaw CJ, Woo D, Adeoye O, Flaherty ML, et al. Towards phenotyping stroke: leveraging data from a large-scale epidemiological study to detect stroke diagnosis. PLoS ONE. (2018) 13:e0192586. doi: 10.1371/journal.pone.0192586
13. Arbabshirani MR, Fornwalt BK, Mongelluzzo GJ, Suever JD, Geise BD, Patel AA, et al. Advanced machine learning in action: identification of intracranial hemorrhage on computed tomography scans of the head with clinical workflow integration. NPJ Digit Med. (2018) 1:9. doi: 10.1038/s41746-017-0015-z
14. Lee H, Lee EJ, Ham S, Lee HB, Lee JS, Kwon SU, et al. Machine learning approach to identify stroke within 45 hours. Stroke. (2020) 51:860–6. doi: 10.1161/STROKEAHA.119.027611
15. Ho KC, Speier W, Zhang H, Scalzo F, El-Saden S, Arnold CW, et al. Machine learning approach for classifying ischemic stroke onset time from imaging. IEEE Trans Med Imaging. (2019) 38:1666–76. doi: 10.1109/TMI.2019.2901445
16. Park E, Lee K, Han T, Nam HS. Automatic grading of stroke symptoms for rapid assessment using optimized machine learning and 4-limb kinematics: clinical validation study. J Med Internet Res. (2020) 22:e20641. doi: 10.2196/20641
17. Kogan E, Twyman K, Heap J, Milentijevic D, Lin JH, Alberts M. Assessing stroke severity using electronic health record data: a machine learning approach. BMC Med Inform Decis Mak. (2020) 20:8. doi: 10.1186/s12911-019-1010-x
18. Chung JW, Kim YC, Cha J, Choi EH, Kim BM, Seo WK, et al. Characterization of clot composition in acute cerebral infarct using machine learning techniques. Ann Clin Transl Neurol. (2019) 6:739–47. doi: 10.1002/acn3.751
19. Dhar R, Chen Y, An H, Lee JM. Application of machine learning to automated analysis of cerebral edema in large cohorts of ischemic stroke patients. Front Neurol. (2018) 9:687. doi: 10.3389/fneur.2018.00687
20. Liu J, Xu H, Chen Q, Zhang T, Sheng W, Huang Q, et al. Prediction of hematoma expansion in spontaneous intracerebral hemorrhage using support vector machine. EBioMedicine. (2019) 43:454–9. doi: 10.1016/j.ebiom.2019.04.040
21. Dengler NF, Madai VI, Unteroberdörster M, Zihni E, Brune SC, Hilbert A, et al. Outcome prediction in aneurysmal subarachnoid hemorrhage: a comparison of machine learning methods and established clinico-radiological scores. Neurosurg Rev. (2021) 44:2837–46. doi: 10.1007/s10143-020-01453-6
22. Ramos LA, van der Steen WE, Sales Barros R, Majoie C, van den Berg R, Verbaan D, et al. Machine learning improves prediction of delayed cerebral ischemia in patients with subarachnoid hemorrhage. J Neurointerv Surg. (2019) 11:497–502. doi: 10.1136/neurintsurg-2018-014258
23. Bentley P, Ganesalingam J, Carlton Jones AL, Mahady K, Epton S, Rinne P, et al. Prediction of stroke thrombolysis outcome using CT brain machine learning. Neuroimage Clin. (2014) 4:635–40. doi: 10.1016/j.nicl.2014.02.003
24. Géron A. Hands-On Machine Learning With Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems. Sebastopol, CA: O'Reilly Media (2019).
25. Feng R, Badgeley M, Mocco J, Oermann EK. Deep learning guided stroke management: a review of clinical applications. J Neurointerv Surg. (2018) 10:358–62. doi: 10.1136/neurintsurg-2017-013355
26. Nielsen A, Hansen MB, Tietze A, Mouridsen K. Prediction of tissue outcome and assessment of treatment effect in acute ischemic stroke using deep learning. Stroke. (2018) 49:1394–401. doi: 10.1161/STROKEAHA.117.019740
27. Yu Y, Xie Y, Thamm T, Gong E, Ouyang J, Huang C, et al. Use of deep learning to predict final ischemic stroke lesions from initial magnetic resonance imaging. J Am Med Assoc Netw Open. (2020) 3:e200772. doi: 10.1001/jamanetworkopen.2020.0772
28. Dey A. Machine learning algorithms: a review. Int J Comp Sci Inform Technol. (2016) 7:1174–9. Available online at: https://ijcsit.com/docs/Volume%207/vol7issue3/ijcsit2016070332.pdf
29. Zhou ZH. Ensemble learning. Encycl Biometr. (2009) 2009:270–3. doi: 10.1007/978-0-387-73003-5_293
30. Géron A. Hands-on Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems. Sebastopol, CA: O'Reilly Media, Inc. (2017).
31. Garcia-Terriza L, Risco-Martin JL, Ayala JL, Rosello GR, Camarasaltas JM. IEEE: comparison of different machine learning approaches to model stroke subtype classification and risk prediction. In: 2019 Spring Simulation Conference. Tuscan, AZ: Springsim. (2019). doi: 10.23919/SpringSim.2019.8732846
32. Sung SF, Lin CY, Hu YH. EMR-based phenotyping of ischemic stroke using supervised machine learning and text mining techniques. IEEE J Biomed Health Informat. (2020) 24:2922–31. doi: 10.1109/JBHI.2020.2976931
33. Giri EP, Fanany MI, Arymurthy AM, Wijaya SK. Ischemic stroke identification based on EEG and EOG using ID convolutional neural network and batch normalization. In: 2016 International Conference on Advanced Computer Science and Information Systems, ICACSIS 2016. Malang (2017). doi: 10.1109/ICACSIS.2016.7872780
34. Takahashi N, Lee Y, Tsai DY, Matsuyama E, Kinoshita T, Ishii K. An automated detection method for the MCA dot sign of acute stroke in unenhanced CT. Radiol Phys Technol. (2014) 7:79–88. doi: 10.1007/s12194-013-0234-1
35. Chen L, Bentley P, Rueckert D. Fully automatic acute ischemic lesion segmentation in DWI using convolutional neural networks. Neuroimage Clin. (2017) 15:633–43. doi: 10.1016/j.nicl.2017.06.016
36. Bouts MJ, Tiebosch IA, van der Toorn A, Viergever MA, Wu O, Dijkhuizen RM. Early identification of potentially salvageable tissue with MRI-based predictive algorithms after experimental ischemic stroke. J Cereb Blood Flow Metab. (2013) 33:1075–82. doi: 10.1038/jcbfm.2013.51
37. Chen Y, Dhar R, Heitsch L, Ford A, Fernandez-Cadenas I, Carrera C, et al. Automated quantification of cerebral edema following hemispheric infarction: application of a machine-learning algorithm to evaluate CSF shifts on serial head CTs. Neuroimage Clin. (2016) 12:673–80. doi: 10.1016/j.nicl.2016.09.018
38. Colak A, Karaman E, Turtay MG. Application of knowledge discovery process on the prediction of stroke. Comput Methods Programs Biomed. (2015) 119:181–5. doi: 10.1016/j.cmpb.2015.03.002
39. Maier O, Schröder C, Forkert ND, Martinetz T, Handels H. Classifiers for ischemic stroke lesion segmentation: a comparison study. PLoS ONE. (2015) 10:e0145118. doi: 10.1371/journal.pone.0145118
40. Öman O, Mäkelä T, Salli E, Savolainen S, Kangasniemi M. 3D convolutional neural networks applied to CT angiography in the detection of acute ischemic stroke. Eur Radiol Exp. (2019) 3:8. doi: 10.1186/s41747-019-0085-6
41. Chen Z, Zhang R, Xu F, Gong X, Shi F, Zhang M, et al. Novel prehospital prediction model of large vessel occlusion using artificial neural network. Front Aging Neurosci. (2018) 10:181. doi: 10.3389/fnagi.2018.00222
42. Dhar R, Falcone GJ, Chen YS, Hamzehloo A, Kirsch EP, Noche RB, et al. Deep learning for automated measurement of hemorrhage and perihematomal edema in supratentorial intracerebral hemorrhage. Stroke. (2020) 51:648–51. doi: 10.1161/STROKEAHA.119.027657
43. Arab A, Chinda B, Medvedev G, Siu W, Guo H, Gu T, et al. A fast and fully-automated deep-learning approach for accurate hemorrhage segmentation and volume quantification in non-contrast whole-head CT. Sci Rep. (2020) 10:7. doi: 10.1038/s41598-020-76459-7
44. Ko A, Chung H, Lee H, Lee J. IEEE: feasible study on intracranial hemorrhage detection and classification using a CNN-LSTM network. In: 42nd Annual International Conference of the IEEE-Engineering-in-Medicine-and-Biology-Society (EMBC). Montreal (2020). doi: 10.1109/EMBC44109.2020.9176162
45. Irene K, Masum MA, Yunus RE, Jatmiko W. Segmentation and approximation of blood volume in intracranial hemorrhage patients based on computed tomography scan images using deep learning method. In: 2020 International Workshop on Big Data and Information Security, IWBIS 2020. Depok (2020). doi: 10.1109/IWBIS50925.2020.9255593
46. Sage A, Badura P. Intracranial hemorrhage detection in head CT using double-branch convolutional neural network, support vector machine, and random forest. Appl Sci. (2020) 10:577. doi: 10.3390/app10217577
47. Ye A, Gao F, Yin YB, Guo DF, Zhao PF, Lu Y, et al. Precise diagnosis of intracranial hemorrhage and subtypes using a three-dimensional joint convolutional and recurrent neural network. Eur Radiol. (2019) 29:6191–201. doi: 10.1007/s00330-019-06163-2
48. Chang PD, Kuoy E, Grinband J, Weinberg BD, Thompson M, Homo R, et al. Hybrid 3D/2D convolutional neural network for hemorrhage evaluation on head CT. Am J Neuroradiol. (2018) 39:1609–16. doi: 10.3174/ajnr.A5742
49. Capoglu S, Savarraj JP, Sheth SA, Choi HA, Giancardo L. Representation learning of 3D brain angiograms, an application for cerebral vasospasm prediction. In: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS. Berlin (2019). doi: 10.1109/EMBC.2019.8857815
50. Tanioka S, Ishida F, Nakano F, Kawakita F, Kanamaru H, Nakatsuka Y, et al. Machine learning analysis of matricellular proteins and clinical variables for early prediction of delayed cerebral ischemia after aneurysmal subarachnoid hemorrhage. Mol Neurobiol. (2019) 56:7128–35. doi: 10.1007/s12035-019-1601-7
51. Giacalone M, Rasti P, Debs N, Frindel C, Cho TH, Grenier E, et al. Local spatio-temporal encoding of raw perfusion MRI for the prediction of final lesion in stroke. Med Image Anal. (2018) 50:117–26. doi: 10.1016/j.media.2018.08.008
52. Grosser M, Gellißen S, Borchert P, Sedlacik J, Nawabi J, Fiehler J, et al. Improved multi-parametric prediction of tissue outcome in acute ischemic stroke patients using spatial features. PLoS ONE. (2020) 15:e0228113. doi: 10.1371/journal.pone.0228113
53. Foroushani HM, Hamzehloo A, Kumar A, Chen Y, Heitsch L, Slowik A, et al. Quantitative serial CT imaging-derived features improve prediction of malignant cerebral edema after ischemic stroke. Neurocrit Care. (2020) 33:785–92. doi: 10.1007/s12028-020-01056-5
54. Yu Y, Guo D, Lou M, Liebeskind D, Scalzo F. Prediction of hemorrhagic transformation severity in acute stroke from source perfusion MRI. IEEE Trans Biomed Eng. (2018) 65:2058–65. doi: 10.1109/TBME.2017.2783241
55. Scalzo F, Alger JR, Hu X, Saver JL, Dani KA, Muir KW, et al. Multi-center prediction of hemorrhagic transformation in acute ischemic stroke using permeability imaging features. Magn Reson Imaging. (2013) 31:961–9. doi: 10.1016/j.mri.2013.03.013
56. van Os HJA, Ramos LA, Hilbert A, van Leeuwen M, van Walderveen MAA, Kruyt ND, et al. Predicting outcome of endovascular treatment for acute ischemic stroke: potential value of machine learning algorithms. Front Neurol. (2018) 9:784. doi: 10.3389/fneur.2018.00784
57. Hilbert A, Ramos LA, van Os HJA, Olabarriaga SD, Tolhuisen ML, Wermer MJH, et al. Data-efficient deep learning of radiological image data for outcome prediction after endovascular treatment of patients with acute ischemic stroke. Comput Biol Med. (2019) 115:103516. doi: 10.1016/j.compbiomed.2019.103516
58. ARondina M, Filippone M, Girolami M, Ward NS. Decoding post-stroke motor function from structural brain imaging. Neuroimage Clin. (2016) 12:372–80. doi: 10.1016/j.nicl.2016.07.014
59. Matsumoto K, Nohara Y, Soejima H, Yonehara T, Nakashima N, Kamouchi M. Stroke prognostic scores and data-driven prediction of clinical outcomes after acute ischemic stroke. Stroke. (2020) 51:1477–83. doi: 10.1161/STROKEAHA.119.027300
60. Scrutinio D, Ricciardi C, Donisi L, Losavio E, Battista P, Guida P, et al. Machine learning to predict mortality after rehabilitation among patients with severe stroke. Sci Rep. (2020) 10:20127. doi: 10.1038/s41598-020-77243-3
61. Ge Y, Wang Q, Wang L, Wu H, Peng C, Wang J, et al. Predicting post-stroke pneumonia using deep neural network approaches. Int J Med Inform. (2019) 132:103986. doi: 10.1016/j.ijmedinf.2019.103986
62. Li X, Wu M, Sun C, Zhao Z, Wang F, Zheng X, et al. Using machine learning to predict stroke-associated pneumonia in Chinese acute ischaemic stroke patients. Eur J Neurol. (2020) 27:1656–63. doi: 10.1111/ene.14295
63. Wang HL, Hsu WY, Lee MH, Weng HH, Chang SW, Yang JT, et al. Automatic machine-learning-based outcome prediction in patients with primary intracerebral hemorrhage. Front Neurol. (2019) 10:910. doi: 10.3389/fneur.2019.00910
64. Heo J, Yoon JG, Park H, Kim YD, Nam HS, Heo JH. Machine learning-based model for prediction of outcomes in acute stroke. Stroke. (2019) 50:1263–5. doi: 10.1161/STROKEAHA.118.024293
65. Lin J, Jiang A, Ling M, Mo Y, Li M, Zhao J. Prediction of neurologic deterioration based on support vector machine algorithms and serum osmolarity equations. Brain Behav. (2018) 8:e01023. doi: 10.1002/brb3.1023
66. Brugnara G, Neuberger U, Mahmutoglu MA, Foltyn M, Herweh C, Nagel S, et al. Multimodal predictive modeling of endovascular treatment outcome for acute ischemic stroke using machine-learning. Stroke. (2020) 51:3541–51. doi: 10.1161/STROKEAHA.120.030287
67. Forkert ND, Verleger T, Cheng B, Thomalla G, Hilgetag CC, Fiehler J. Multiclass support vector machine-based lesion mapping predicts functional outcome in ischemic stroke patients. PLoS ONE. (2015) 10:e0129569. doi: 10.1371/journal.pone.0129569
68. Monteiro M, Fonseca AC, Freitas AT, Pinho EMT, Francisco AP, Ferro JM, et al. Using machine learning to improve the prediction of functional outcome in ischemic stroke patients. IEEE/ACM Trans Comput Biol Bioinform. (2018) 15:1953–9. doi: 10.1109/TCBB.2018.2811471
69. Jang SK, Chang JY, Lee JS, Lee EJ, Kim YH, Han JH, et al. Reliability and clinical utility of machine learning to predict stroke prognosis: comparison with logistic regression. J Stroke. (2020) 22:403–6. doi: 10.5853/jos.2020.02537
70. Hope TM, Seghier ML, Leff AP, Price CJ. Predicting outcome and recovery after stroke with lesions extracted from MRI images. Neuroimage Clin. (2013) 2:424–33. doi: 10.1016/j.nicl.2013.03.005
71. Lopes R, Bournonville C, Kuchcinski G, Dondaine T, Mendyk AM, Viard R, et al. Prediction of long-term cognitive functions after minor stroke, using functional connectivity. Neurology. (2021) 96:e1167–79. doi: 10.1212/WNL.0000000000011452
72. Sale P, Ferriero G, Ciabattoni L, Cortese AM, Ferracuti F, Romeo L, et al. Predicting motor and cognitive improvement through machine learning algorithm in human subject that underwent a rehabilitation treatment in the early stage of stroke. J Stroke Cerebrovasc Dis. (2018) 27:2962–72. doi: 10.1016/j.jstrokecerebrovasdis.2018.06.021
73. Iwamoto Y, Imura T, Tanaka R, Imada N, Inagawa T, Araki H, et al. Development and validation of machine learning-based prediction for dependence in the activities of daily living after stroke inpatient rehabilitation: a decision-tree analysis. J Stroke Cerebrovasc Dis. (2020) 29:105332. doi: 10.1016/j.jstrokecerebrovasdis.2020.105332
74. Lin WY, Chen CH, Tseng YJ, Tsai YT, Chang CY, Wang HY, et al. Predicting post-stroke activities of daily living through a machine learning-based approach on initiating rehabilitation. Int J Med Inform. (2018) 111:159–64. doi: 10.1016/j.ijmedinf.2018.01.002
75. Tozlu A, Edwards D, Boes A, Labar D, Tsagaris KZ, Silverstein J, et al. Machine learning methods predict individual upper-limb motor impairment following therapy in chronic stroke. Neurorehabil Neural Repair. (2020) 34:428–39. doi: 10.1177/1545968320909796
76. Stinear CM, Barber PA, Petoe M, Anwar S, Byblow WD. The PREP algorithm predicts potential for upper limb recovery after stroke. Brain. (2012) 135:2527–35. doi: 10.1093/brain/aws146
77. Murray NM, Unberath M, Hager GD, Hui FK. Artificial intelligence to diagnose ischemic stroke and identify large vessel occlusions: a systematic review. J Neurointerv Surg. (2020) 12:156–64. doi: 10.1136/neurintsurg-2019-015135
78. Mokli Y, Pfaff J, Dos Santos DP, Herweh C, Nagel S. Computer-aided imaging analysis in acute ischemic stroke - background and clinical applications. Neurol Res Pract. (2019) 1:23. doi: 10.1186/s42466-019-0028-y
79. Straka MG, Albers W, Bammer R. Real-time diffusion-perfusion mismatch analysis in acute stroke J Magn Reson Imaging. (2010) 32:1024–37. doi: 10.1002/jmri.22338
80. Lansberg MG, Straka M, Kemp S, Mlynash M, Wechsler LR, Jovin TG, et al. MRI profile and response to endovascular reperfusion after stroke (DEFUSE 2): a prospective cohort study. Lancet Neurol. (2012) 11:860–7. doi: 10.1016/S1474-4422(12)70203-X
81. Justin P. FDA approves stroke-detecting AI software. Nat Biotechnol. (2018) 36:290. doi: 10.1038/nbt0418-290
82. Yahav-Dovrat A, Saban M, Merhav G, Lankri I, Abergel E, Eran A, et al. Evaluation of artificial intelligence-powered identification of large-vessel occlusions in a comprehensive stroke center. Am J Neuroradiol. (2021) 42:247–54. doi: 10.3174/ajnr.A6923
83. GBD 2016 Causes of Death Collaborators. Global, regional, and national age-sex specific mortality for 264 causes of death:1980–2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet. (2017) 390:1151–210. doi: 10.1016/S0140-6736(17)32152-9
84. Stevens RD, Sutter R. Prognosis in severe brain injury. Crit Care Med. (2013) 41:1104–23. doi: 10.1097/CCM.0b013e318287ee79
85. Holloway RG, Arnold RM, Creutzfeldt CJ, Lewis EF, Lutz BJ, McCann RM, et al. Palliative and end-of-life care in stroke: a statement for healthcare professionals from the American Heart Association/American Stroke Association. Stroke. (2014) 45:1887–916. doi: 10.1161/STR.0000000000000015
86. Morgenstern LB, Zahuranec DB, Sánchez BN, Becker KJ, Geraghty M, Hughes R, et al. Full medical support for intracerebral hemorrhage. Neurology. (2015) 84:1739–44. doi: 10.1212/WNL.0000000000001525
87. AHwang Y, Dell CA, Sparks MJ, Watson TD, Langefeld CD, Comeau ME, et al. Clinician judgment vs. formal scales for predicting intracerebral hemorrhage outcomes. Neurology. (2016) 86:126–33. doi: 10.1212/WNL.0000000000002266
88. Hemphill JC 3rd, Bonovich DC, Besmertis L, Manley GT, Johnston SC. The ICH score: a simple, reliable grading scale for intracerebral hemorrhage. Stroke. (2001) 32:891–7. doi: 10.1161/01.STR.32.4.891
89. Nawabi J, Kniep H, Elsayed S, Friedrich C, Sporns P, Rusche T, et al. Imaging-based outcome prediction of acute intracerebral hemorrhage. Transl Stroke Res. (2021). doi: 10.1007/s12975-021-00891-8
90. Sennfält S, Pihlsgård M, Petersson J, Norrving B, Ullberg T. Long-term outcome after ischemic stroke in relation to comorbidity - an observational study from the Swedish Stroke Register (Riksstroke). Eur Stroke J. (2020) 5:36–46. doi: 10.1177/2396987319883154
91. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, et al. ImageNet large scale visual recognition challenge. Int J Comput Vis. (2015) 115:211–52. doi: 10.1007/s11263-015-0816-y
92. Chan S, Siegel EL. Will machine learning end the viability of radiology as a thriving medical specialty? Br J Radiol. (2019) 92:20180416. doi: 10.1259/bjr.20180416
93. ABeam L, Kohane IS. Big data and machine learning in health care. J Am Med Assoc. (2018) 319:1317–8. doi: 10.1001/jama.2017.18391
94. SLiebeskind S, Scalzo Editorial F. Machine learning and decision support in stroke. Front Neurol. (2020) 11:486. doi: 10.3389/fneur.2020.00486
95. Vayena A, Blasimme A, Cohen IG. Machine learning in medicine: addressing ethical challenges. PLoS Med. (2018) 15:e1002689. doi: 10.1371/journal.pmed.1002689
96. Obermeyer Z, Emanuel EJ. Predicting the future - big data, machine learning, and clinical medicine. N Engl J Med. (2016) 375:1216–9. doi: 10.1056/NEJMp1606181
97. Marra A, Ely EW, Pandharipande PP, Patel MB. The ABCDEF bundle in critical care. Crit Care Clin. (2017) 33:225–43. doi: 10.1016/j.ccc.2016.12.005
98. Cramer SC, Le V, Saver JL, Dodakian L, See J, Augsburger R, et al. Intense arm rehabilitation therapy improves the modified rankin scale score: association between gains in impairment and function. Neurology. (2021) 96:e1812–22. doi: 10.1212/WNL.0000000000011667
Keywords: machine learning, artificial intelligence, deep learning, stroke diagnosis, stroke prognosis, stroke outcome prediction, machine learning in medical imaging, machine learning in medicine
Citation: Mainali S, Darsie ME and Smetana KS (2021) Machine Learning in Action: Stroke Diagnosis and Outcome Prediction. Front. Neurol. 12:734345. doi: 10.3389/fneur.2021.734345
Received: 01 July 2021; Accepted: 28 October 2021;
Published: 06 December 2021.
Edited by:
Jiang Li, Geisinger Medical Center, United StatesReviewed by:
Feifei Ma, Vall d'Hebron Research Institute (VHIR), SpainHarshawardhan Deshpande, National Institute on Drug Abuse (NIDA), United States
Copyright © 2021 Mainali, Darsie and Smetana. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Shraddha Mainali, shraddha.mainali@vcuhealth.org
†These authors have contributed equally to this work