Automated identification of thrombectomy amenable vessel occlusion on computed tomography angiography using deep learning

Introduction: We developed and externally validated a fully automated algorithm using deep learning to detect large vessel occlusion (LVO) in computed tomography angiography (CTA).

Method: A total of 2,045 patients with acute ischemic stroke who underwent CTA were included in the development of our model. We validated the algorithm using two separate external datasets: one with 64 patients (external 1) and another with 313 patients (external 2), with ischemic stroke. In the context of current clinical practice, thrombectomy amenable vessel occlusion (TAVO) was defined as an occlusion in the intracranial internal carotid artery (ICA), or in the M1 or M2 segment of the middle cerebral artery (MCA). We employed the U-Net for vessel segmentation on the maximum intensity projection images, followed by the application of the EfficientNetV2 to predict TAVO. The algorithm’s diagnostic performance was evaluated by calculating the area under the receiver operating characteristics curve (AUC), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).

Results: The mean age in the training and validation dataset was 68.7 ± 12.6; 56.3% of participants were men, and 18.0% had TAVO. The algorithm achieved AUC of 0.950 (95% CI, 0.915–0.971) in the internal test. For the external datasets 1 and 2, the AUCs were 0.970 (0.897–0.997) and 0.971 (0.924–0.990), respectively. With a fixed sensitivity of 0.900, the specificities and PPVs for the internal test, external test 1, and external test 2 were 0.891, 0.796, and 0.930, and 0.665, 0.583, and 0.667, respectively. The algorithm demonstrated a sensitivity and specificity of approximately 0.95 in both internal and external datasets, specifically for cases involving intracranial ICA or M1-MCA occlusion. However, the diagnostic performance was somewhat reduced for isolated M2-MCA occlusion; the AUC for the internal and combined external datasets were 0.903 (0.812–0.944) and 0.916 (0.816–0.963), respectively.

Conclusion: We developed and externally validated a fully automated algorithm that identifies TAVO. Further research is needed to evaluate its effectiveness in real-world clinical settings. This validated algorithm has the potential to assist early-career physicians, thereby streamlining the treatment process for patients who can benefit from endovascular treatment.

Introduction

Advancements in stroke imaging and procedural devices have extended the endovascular treatment (EVT) window for patients with hyperacute ischemic stroke (1, 2). The DAWN (DWI or CTP Assessment with Clinical Mismatch in the Triage of Wake-Up and Late Presenting Strokes Undergoing Neurointervention with Trevo) and DFFUSE 3 (Endovascular Therapy Following Imaging Evaluation for Ischemic Stroke) trials have changed the standard of care for ischemic stroke patients who present within 6 to 24 h of their last well known status.

Triage entry for clinical trials primarily relies on magnetic resonance imaging (MRI) or computed tomography perfusion (CTP) to identify clinical or tissue mismatch and is now endorsed in guidelines. However, access to acute MRI or CTP is limited and not commonly available in the majority of primary stroke centers worldwide. Recent studies have brought attention to more readily available imaging techniques, such as CT angiography (3). The CT for Late Endovascular Reperfusion (CLEAR) trial (4) revealed no significant differences in clinical outcomes between patients selected using non-contrast CT with CT angiography and those selected using CTP or MRI. In addition, a sub-study (5) of the HERMES collaboration (Highly Effective Reperfusion Evaluated in Multiple Endovascular Stroke Trials) has extended this approach to the early time window (0–6 h) by demonstrating that the rates of favorable functional outcomes were comparable between patients who underwent CTP and those who did not.

Two-thirds of EVT candidates were initially routed to centers not equipped for EVT (6), despite better outcomes and higher chances of receiving EVT at EVT-capable centers. Consequently, non-EVT-capable centers must consistently identify large vessel occlusion (LVO) around the clock, ensuring quick reporting to facilitate patient transfer to EVT-capable centers. However, a lack of vascular specialists poses challenges for many smaller, non-EVT-capable centers. Even in EVT-capable centers, the ability to screen CTAs for the presence of LVOs can streamline workflow, staffing, and door-to-puncture times by facilitating LVO detection. Machine learning has been employed to automate LVO detection in CTA, which is now in clinical use in a few countries (7, 8). However, independent external evaluation of these automated LVO detection algorithms have shown only modest sensitivity (7, 8). Moreover, machine learning algorithms contingent on Hounsfield unit and hemisphere asymmetry information have limited its applicability in patients with bilateral occlusions, such as Moyamoya disease (9).

In contrast to initial focus on intracranial LVO, recent advancements in neurointerventional devices and cerebrovascular imaging have expanded the application of EVT to medium vessel occlusions. Consequently, the selection of appropriate EVT candidates has become more complex, necessitating advanced imaging. This complexity poses a challenge for early-career physicians in making treatment decision (10).

In the context of current clinical practice, we have developed and validated a fully automated deep learning algorithm to detect comprehensive EVT target vessels. This includes not only the well-known LVOs but also other relevant vessel occlusions. We utilized a dataset from multiple centers in Korea, encompassing 2,441 patients with acute ischemic stroke. The deep learning algorithms were specifically designed for (1) selecting the appropriate slices for consistent maximal intensity projection (MIP) image generation, (2) segmenting vessels on MIP images, and (3) identifying vessel occlusions using the vessel segmentation mask. We further validated the algorithm with two independent external datasets.

Materials and methods

Datasets

Training validation and internal test

From May 2011 and June 2013, 1,745 patients were collected from two hospitals. Additionally, 389 patients were included from a university hospital between August 2020 and May 2021. After combining these groups, a total of 2,134 patients who were suspected with ischemic stroke and who underwent CT angiography were initially considered (refer to Supplemental Figure 1). Following the exclusion of 89 patients, the final cohort consisted of 2,045 patients, who were randomly divided into training (1,277), validation (144), and internal test (624) datasets with a 63:7:30 ratio.

External test

Between April 2011 and July 2013, a total of 71 patients were included from two tertiary hospitals. After excluding 7 patients, 64 were assigned to external test dataset 1. In addition, 337 patients from a university hospital were included for the period between February 2017 and March 2022. After excluding 24 patients from this group, 313 were assigned to external test dataset 2. The study procedure was approved by the institutional review boards of each hospital, and written consent was waived due to the retrospective and anonymised nature of the study design.

Definition of thrombectomy amenable vessel occlusion

In our study, the vascular occlusions we aimed to investigate are termed as thrombectomy amenable vessel occlusion (TAVO) which include any arterial occlusion involving the intracranial internal carotid artery (ICA), middle cerebral artery (MCA)-M1, and MCA-M2 segments. Intracranial ICA is defined as the segment of the ICA from the petrous part to the MCA-ACA (anterior cerebral artery) bifurcation. MCA-M1 indicates the MCA segment from the MCA-ACA bifurcation to the MCA branching point, and the MCA-M2 refers to the segment of the MCA ascending vertically along with Sylvian fissure from its branching point. For the subsequent analysis, TAVO patients were divided into intracranial LVO and isolated MCA-M2 occlusion in our study (11). In cases of early division of the MCA, a functional rather than traditional angiographic definition was adopted; the short proximal trunk was called the M1 segment and the branches distal to division were defined as M2 segments. To determine the presence or absence of TAVO (ground truth label), each image used in the study was reviewed by an experienced neurologist along with the subject’s MR image (MRI) scans and patient symptom data. The neurologist’s TAVO diagnosis was cross-referenced with the stroke registry, which was independently verified by attending vascular neurologists at each hospital. In case of disagreement, a consensus was made.

Algorithm description

Slices selection for maximal intensity projection image generation

In order to maintain the uniformity of the input image for the deep learning model, we have developed an automated approach for selecting slices from the source images (Figure 1). These slices are then used to create a maximum intensity projection (MIP) image. We constructed a sagittal bone array by summing the pixel values of the source images. This process automatically outlines the skull, enabling the automatic determination of the vertex and the C1 atlas (12). Using this method on the training data, we effectively identified target slices in 1,240 cases, which accounts for 97.1% of the total. We calculated the mean and standard deviation of the distance between the vertex and C1 atlas in these patients. For cases falling outside the mean ± 2SD range, we utilized the second method, a deep learning-based algorithm. This model, modified Inception (13), classifies CTA source images from the vertex to the C1 atlas as target regions. We used 100 randomly selected cases (4,020 CTA source images), where the first method correctly identified the vertex and C1 atlas. Of these, 89 cases were used for training and 11 cases for validation. The method accurately identified target slices within a 5% margin of error in all cases in the validation dataset. By combining these two techniques, we successfully produced MIP images with a uniform range across all patients in the training and validation dataset. Following skull stripping using our in-house algorithm, we created axial MIP images suitable for the deep learning method, utilizing software developed at our institution (14).

Figure 1

Figure 1. Depiction of the algorithm used to detect automatic thrombectomy amenable vessel occlusion (TAVO). (I) the acquisition of unprocessed CT angiogram images in DICOM (Digital Imaging and Communications in Medicine) format, accompanied by the automated selection of slices from the vertex and C1 atlas. (II) Skull removal, standard template registration, and maximal intensity projection (MIP) image rendering. (III) Using a rendered axial MIP image to segment blood vessels using deep learning (DL) and merging the blood vessel masks. (IV) Predicting TAVO using the merged blood vessel mask and generating a heatmap that identifies the region that influences the DL decision the most.

Vessel segmentation on maximal intensity projection image

We developed a 2D U-Net based on the Inception Module specifically for vessel segmentation in axial MIP images (15). This model was trained to segment vessels from the generated MIP images. The U-Net architecture integrates structural information from the network with the semantic information from the Inception Module, enabling more precise segmentation of vessels in MIP images. After generating MIP images, researchers manually segmented intracranial arteries in 208 randomly selected patients (16% of whom had TAVO) from the training (n = 189) and validation dataset (n = 19). For the training of the vessel segmentation model, we manually outlined all discernible anterior circulation intracranial arteries on MIP, including the distal ICA, MCA, ACA, posterior cerebral artery, and their visible branches. This manual segmentation was conducted under the supervision of an experienced vascular neurologist (W-S Ryu). The trained model achieved a Dice similarity coefficient of 0.80, indicating strong agreement with the manual segmentation performed by researchers supervised by a vascular neurologist. Two representative cases comparing automated intracranial vessel segmentation with manual segmentation is provided in Supplementary Figure 2.

Vessel occlusion detection algorithm

After automatically segmenting blood vessels on the axial MIP image, the vessel masks from each slice were combined to create a two-dimensional compressed image of the vessel masks. These compressed images serve as inputs for our deep learning model, trained for TAVO classification. We observed that using multiple compressed images of vessel masks at constant intervals as input significantly reduced model’s performance compared to using a single compressed image. This was primarily due to overfitting of the algorithm on MCA slices where most TAVOs are located in a specific slice of the 3D volume. The issue of overfitting became more pronounced with an increase in the number of compressed images. Therefore, to mitigate this overfitting, we compressed the segmented vessel masks into a single image for this study. EfficientNetV2 was employed to train the TAVO classification model (16). Data augmentation was applied during the training process to prevent overfitting and alleviate domain shift problems. The augmentation algorithm was implemented using albumentations, a Python library for image augmentations (17). A batch size of 32 was maintained, with TAVO and non-TAVO cases sampled at a 1:1 ratio in each batch. For the deep learning training, intracranial LVO and isolated MCA-M2 occlusion were combined into a single category as TAVO.

For the training process, we used the AdamW optimizer with a batch size of 32, and a StepLR learning rate Scheduler with a step size of 7 and gamma of 0.1. Additionally, we addressed class imbalance by using a Weighted Random Sampler to sample the TAVO cases more frequently. The training utilized libraries including Python, PyTorch, TensorFlow, Pydicom, OpenCV, ITK, and was performed on an NVIDIA RTX A6000 GPU. The developed software operates on Window 10 or higher.

Statistical analysis

Data were presented as mean ± standard deviation, median (interquartile range), or number (percentage). To compare baseline characteristics between training and validation, internal test, external test 1, external test 2, we employed ANOVA or Kruskal-Wallis test for continuous variables and chi-square test or Fisher exact test, as appropriate. The diagnostic performance of the algorithm for detecting TAVO was assessed using sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV), calculated through receiver operating characteristics (ROC) analysis. The Youden index was utilized to determine the optimal threshold (18). We then stratified the TAVO patients into intracranial LVO and isolated MCA-M2 occlusion groups and conducted two subgroup analyses: one excluding subjects with isolated MCA-M2 occlusion and another excluding intracranial LVO. In this subgroup analysis, external test 1 and external test 2 dataset were combined due to small number of subjects in the external test 1 dataset. The DeLong test was used to calculate 95% confidence interval of the area under the curve (AUC) (19). For other parameters, bootstrap analysis with 1,000 repetitions was conducted to calculate 95% confidence intervals. All statistical analyses were performed using STATA 16.0 (STATA Corp., Texas, United States), with p < 0.05 considered statistically significant.

Results

Baseline characteristics

The mean ages for the training and validation, internal test, external test 1, and external test 2 were 68.7, 68.3, 68.8, and 67.1 years, respectively, as shown in Table 1. The prevalence of male, atrial fibrillation, and history of prior stroke were similar across all groups. However, the occurrence of TAVO was less frequent in the external test 2 dataset compared to the others (13.4% vs. 18.0 to 23.4%). Notably, the CT vendors and imaging parameters, including slick thickness and pixel spacing, significantly varied between the groups (see Supplementary Table 1). For 100 randomly selected cases from the internal test dataset, the mean processing time from the input of source images to the output of results was 178 ± 11 s.

Table 1

Table 1. Baseline characteristics of training and validation, internal test, and external test datasets.

Diagnostic performance for overall TAVO

Representative examples of TAVO detection using deep learning in four patients with intracranial LVO or isolated MCA-M2 occlusion are illustrated in Figure 2. In the internal test dataset, the deep learning algorithm achieved an area under the AUC of 0.950 (95% CI, 0.915–0.971, see Figure 3). For the external test datasets 1 and 2, the AUCs were 0.970 (0.897–0.997) and 0.971 (0.924–0.990), respectively. Using a cutoff threshold of 0.5, the sensitivity ranged from 0.800 to 0.860 (Table 2), and specificity varied from 0.956 to 1.000. The result of the combined external test dataset is visualized in Supplementary Figure 3. Density plots of TAVO probability demonstrated that the deep learning algorithm could robustly differentiate between TAVO and non-TAVO in both internal and external datasets (as shown in Supplementary Figures 4A,B).

Figure 2

Figure 2. Representative cases for deep learning based thrombectomy amenable vessel occlusion (TAVO) detection. (A) 71-years old women with cervical internal carotid artery (ICA) occlusion (red arrows) without reconstruction of distal flow. (B) 53-years old man with left distal carotid and left proximal middle cerebral artery (MCA)-M1 occlusion (blue arrows). (C) 77-years old women with bilateral proximal MCA-M1 occlusion (green arrows). (D) 68-years old women with occlusion of the proximal left inferior M2 division (yellow arrows). In all cases, the heatmap visualize the occlusion site or a paucity of distal flow.

Figure 3

Figure 3. ROC analysis. ROC curves for detection of thrombectomy amenable vessel occlusion (TAVO) occlusions. (A) Internal test. (B) External test 1. (C) External test 2. Red dots indicate optimal cutoff points with the maximum Youden index. ROC, receiver operating characteristics; AUC, area under the curve.

Table 2

Table 2. Diagnostic performance of deep learning algorithm detecting TAVO.

The Youden indices in the internal test, external test 1, and external test datasets were 0.824, 0.826, and 0.855, respectively, as detailed in Table 2. In addition, the Youden indices remained stable across a wide range of cutoff points (Supplementary Figures 4C,D). At the optimal cutoff points, the sensitivities recorded were 0.860, 0.867, and 0.929 in the internal, external 1, and external 2 datasets, respectively, with corresponding specificities of 0.964, 0.959, and 0.926. When the external test datasets were combined, at the optimal cutoff point, sensitivity, specificity, PPV, and NPV were 0.895 (0.785–0.960), 0.934 (0.901–0.959), 0.708 (0.589–0.810), and 0.980 (0.958–0.993), respectively (see Supplementary Table 2). With a fixed sensitivity of 0.900, the specificities and PPVs for the internal test, external test 1, and external test 2 were 0.891, 0.796, and 0.930 and 0.665, 0.583, and 0.667, respectively.

Diagnostic performance for intracranial large vessel occlusion

Excluding subjects with isolated MCA-M2 occlusion from the analysis significantly improved the performance of the deep learning algorithms. The AUC increased up to 0.967 in the internal dataset and to 0.993 in the combined external datasets, as shown in Table 3 and Supplementary Figure 5. With a cutoff point of 0.5, the sensitivity and specificity in the internal test were 0.943 (0.872–0.981) and 0.956 (0.935–0.972), respectively. The corresponding values in the combined external dataset were 0.947 (0.823–0.994) and 0.975 (0.951–0.989).

Table 3

Table 3. Diagnostic performance of deep learning algorithm stratified by location of occlusion.

Diagnostic performance for isolated MCA-M2 occlusion

When subjects with intracranial LVO were excluded from the analysis, the diagnostic performance of deep learning algorithm was somewhat diminished. The AUC of internal and combined external datasets were 0.903 (0.812–0.944) and 0.916 (0.816–0.963), respectively. The sensitivities were 0.636 in the internal test and 0.632 in the combined external dataset. However, the specificities remained high at 0.956 and 0.975, respectively.

False positive and false negative samples in external test datasets

In the combined external test datasets, eight cases were incorrectly classified as TAVO (Supplementary Table 3). Among these, three patients had occlusions in the MCA-M3 or MCA-M4 segments, which are not considered candidate for EVT. Of the nine patients misclassified as non-TAVO, seven had isolated occlusions in the MCA-M2 segment. Two patients with false negative results and intracranial LVO had marginal TAVO probability score of 0.155 and 0.355, respectively.

Discussion

In this study, we developed a fully automated deep learning algorithm to detect intracranial anterior circulation arterial occlusions in CTA, which are likely candidates for EVT in hyperacute ischemic stroke. The algorithm underwent external validation in two different datasets and demonstrated high diagnostic sensitivity and specificity. When analyzing occlusion sites separately, the algorithm exhibited an excellent diagnostic performance for intracranial LVO. Although the performance for isolated MCA-M2 occlusion was slightly lower than for intracranial LVO, it remained competitive.

Until recently, a number of studies have reported on AI detection of LVO or TAVO (see Supplementary Table 4). However, these algorithms were either lacked external validation in previous research, or if validated, it was in a limited number of cases (20–23). Moreover, earlier studies reported that the artificial intelligence algorithms achieved AUC scores ranging from 0.74 to 0.86, which may not sufficiently support early-career physicians (20–23). Notably, in external validation sets with adequate sample sizes, our deep learning algorithm achieved AUCs of 0.961, 0.993, and 0.913 for total TAVO, intracranial LVO, and isolated MCA-M2 occlusion, respectively.

The ability of our deep learning algorithm to accurately predict isolated MCA-M2 occlusion with good performance is of significant importance. Although isolated MCA-M2 occlusion is an important and emerging target of the EVT (24), it was reported that the rate of misdiagnosis for isolated MCA-M2 occlusion is substantially higher than that for intracranial LVO even among the neuroimaging specialists (35.0% vs. 9.7%, respectively) (25), and the use of previously developed deep learning models has yielded worse results (50.8%) (20). This characteristic of isolated MCA-M2 occlusion have influenced our model as well, leading to improved specificity and NPV compared to previous studies (20, 21), despite with somewhat less satisfactory sensitivity and PPV. This issue was largely due to false negatives in cases of short-segment (where collaterals reconstituted the M2 segment immediately distal to the occlusion) and incomplete (with antegrade flow) occlusions, where the reduction in the inter-hemispheric vessel density was too small for the algorithm to detect. However, in terms of TAVO detection, these occlusions missed due to false negative may not be ideal candidates for emergent intra-arterial thrombectomy (26).

Our study may have a few potential clinical implications. First, our deep learning model can provide decent assistance to healthcare professionals who may not have much experience with ischemic strokes. Regardless of the reason for admission, the majority of ischemic strokes are initially encountered by non-specialized physicians (27). In such scenarios, identifying vascular occlusion and its location in brain imaging is a complex and demanding task, potentially delaying appropriate stroke treatment. Second, considering that our model processes vascular images in less than 180 s, it could significantly reduce the time taken to make EVT decision. In clinical practice, formal interpretation of brain imaging often requires several hours, and sometimes even more than a day (28). Therefore, rapid primary interpretation of vascular status through our model can play a crucial role in shortening the time to initiate EVT. Third, the objective and reproducible nature of our artificial intelligence software can aid in refining EVT process. This includes the preparation protocol for interventional devices based on occlusion patterns, which is another important factor in reducing reperfusion time.

Limitation of the study

One limitation of our study is the relatively small dataset size used for training the deep learning algorithm. This is particularly true for isolated MCA-M2 occlusion, where fewer cases were collected compared to intracranial LVO, influencing the sensitivity and PPV performance of the model. The second caveat is that this study was conducted within a single country and given the higher prevalence of intracranial atherosclerosis in East Asian populations compared to Western ones, the assessment of the complete occlusion over pre-existing stenosis may have been challenging (29). Such factors could contribute to model’s less favorable performance, underscoring the need for further research involving diverse ethnic groups. Therefore, radiologists and neurologists should be aware of the potential and causes of false negatives in the algorithm’s output. Third, our algorithm does not include neck CT angiography and, therefore, is unable to detect tandem lesions in the cervical ICA.

Future directions

Collateral score and collateral status on CTA serve as important tools in evaluating collateral circulation in patients with TAVO (26). In CTA in patients with TAVO, collateral score and collateral status are used as important parameters to evaluate cerebral hemodynamics and tissue perfusion. These assessments help understand the adequacy of compensatory collateral circulation, which is essential for determining treatment strategies and predicting tissue viability in patients with ischemic stroke. Our developed system can estimate the course of collateral circulation by detecting and displaying the location of TAVO as a heat map. Collateral status involves a comprehensive assessment of the collateral circulation, considering factors such as collateral filling rate, extent of collateral vessels, and final tissue perfusion achieved through collateral flow (30). This assessment plays a pivotal role in predicting the likelihood of ischemic tissue salvage. Appropriate adjuvants contribute to increasing the likelihood of viable tissue despite vascular occlusion, potentially influencing treatment decisions such as endovascular reperfusion therapy. Therefore, auxiliary evaluation of collateral status may be considered in future studies.

Conclusion

We developed and validated a novel, fully automated deep learning algorithm derived from CTA to detect vascular occlusion suitable for EVT. While the algorithm could benefit from further improvements and real-world clinical evaluations, its potential as a tool to assist in the diagnosis of acute ischemic stroke in patients through detection of TAVO has been firmly established.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary material, further inquiries can be directed to the corresponding authors.

Ethics statement

The studies involving humans were approved by Korea University Guro Hospital. The studies were conducted in accordance with the local legislation and institutional requirements. Written informed consent for participation was not required from the participants or the participants’ legal guardians/next of kin in accordance with the national legislation and institutional requirements.

Author contributions

JH: Conceptualization, Investigation, Resources, Writing – original draft. SH: Software, Writing – review & editing. HL: Formal analysis, Software, Writing – original draft. G-HP: Data curation, Software, Writing – original draft. HH: Software, Writing – original draft. DK: Formal analysis, Funding acquisition, Project administration, Software, Writing – original draft. JK: Conceptualization, Data curation, Project administration, Resources, Validation, Writing – review & editing. J-TK: Conceptualization, Data curation, Project administration, Resources, Writing – review & editing. LS: Data curation, Formal analysis, Project administration, Supervision, Validation, Writing – review & editing. CK: Conceptualization, Data curation, Project administration, Resources, Supervision, Validation, Writing – original draft, Writing – review & editing. W-SR: Conceptualization, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Writing – original draft, Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This study was supported by the Multiministry Grant for Medical Device Development (KMDF_PR_20200901_0098) funded by the Korean government.

Conflict of interest

SH, HL, G-HP, HH, DK, and W-SR are employees of JLK Inc.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The author(s) declared that they were an editorial board member of Frontiers, at the time of submission. This had no impact on the peer review process and the final decision.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fneur.2024.1442025/full#supplementary-material

References

1. Albers, GW, Marks, MP, Kemp, S, Christensen, S, Tsai, JP, Ortega-Gutierrez, S, et al. Thrombectomy for stroke at 6 to 16 hours with selection by perfusion imaging. N Engl J Med. (2018) 378:708–18. doi: 10.1056/NEJMoa1713973

PubMed Abstract | Crossref Full Text | Google Scholar

2. Nogueira, RG, Jadhav, AP, Haussen, DC, Bonafe, A, Budzik, RF, Bhuva, P, et al. Thrombectomy 6 to 24 hours after stroke with a mismatch between deficit and infarct. N Engl J Med. (2018) 378:11–21. doi: 10.1056/NEJMoa1706442

PubMed Abstract | Crossref Full Text | Google Scholar

3. Hill, MD, Warach, S, and Rostanski, SK. Should primary stroke centers perform advanced imaging? Stroke. (2022) 53:1423–30. doi: 10.1161/STROKEAHA.121.033528

Crossref Full Text | Google Scholar

4. Nguyen, TN, Abdalkader, M, Nagel, S, Qureshi, MM, Ribo, M, Caparros, F, et al. Noncontrast computed tomography vs computed tomography perfusion or magnetic resonance imaging selection in late presentation of stroke with large-vessel occlusion. JAMA Neurol. (2022) 79:22–31. doi: 10.1001/jamaneurol.2021.4082

PubMed Abstract | Crossref Full Text | Google Scholar

5. Jadhav, AP, Goyal, M, Ospel, J, Campbell, BC, Majoie, CB, Dippel, DW, et al. Thrombectomy with and without computed tomography perfusion imaging in the early time window: a pooled analysis of patient-level data. Stroke. (2022) 53:1348–53. doi: 10.1161/STROKEAHA.121.034331

PubMed Abstract | Crossref Full Text | Google Scholar

6. Kang, J, Kim, S-E, Park, H-K, Cho, Y-J, Kim, JY, Lee, K-J, et al. Routing to endovascular treatment of ischemic stroke in Korea: recognition of need for process improvement. J Korean Med Sci. (2020) 35:e347. doi: 10.3346/jkms.2020.35.e347

Crossref Full Text | Google Scholar

7. Werdiger, F, Gotla, S, Visser, M, Kolacz, J, Yogendrakumar, V, Beharry, J, et al. Automated occlusion detection for the diagnosis of acute ischemic stroke: a detailed performance review. Eur J Radiol. (2023) 164:110845. doi: 10.1016/j.ejrad.2023.110845

PubMed Abstract | Crossref Full Text | Google Scholar

8. Luijten, SP, Wolff, L, Duvekot, MH, van Doormaal, P-J, Moudrous, W, Kerkhoff, H, et al. Diagnostic performance of an algorithm for automated large vessel occlusion detection on CT angiography. J Neurointervention Surg. (2022) 14:794–8. doi: 10.1136/neurintsurg-2021-017842

PubMed Abstract | Crossref Full Text | Google Scholar

9. Schlossman, J, Ro, D, Salehi, S, Chow, D, Yu, W, Chang, PD, et al. Head-to-head comparison of commercial artificial intelligence solutions for detection of large vessel occlusion at a comprehensive stroke center. Front Neurol. (2022) 13:2315. doi: 10.3389/fneur.2022.1026609

Crossref Full Text | Google Scholar

10. Rai, AT, Domico, JR, Buseman, C, Tarabishy, AR, Fulks, D, Lucke-Wold, N, et al. A population-based incidence of M2 strokes indicates potential expansion of large vessel occlusions amenable to endovascular therapy. J NeuroIntervention Surg. (2018) 10:510–5. doi: 10.1136/neurintsurg-2017-013371

PubMed Abstract | Crossref Full Text | Google Scholar

11. Goyal, M, Menon, BK, Krings, T, Patil, S, Qazi, E, McTaggart, RA, et al. What constitutes the M1 segment of the middle cerebral artery? J NeuroIntervention Surg. (2016) 8:1273–7. doi: 10.1136/neurintsurg-2015-012191

PubMed Abstract | Crossref Full Text | Google Scholar

12. Hocking, T, Rigaill, G, Vert, J-P, and Bach, F. Learning sparse penalties for change-point detection using max margin interval regression. In: International Conference on Machine Learning; (2013): PMLR.

Google Scholar

13. Szegedy, C, Ioffe, S, Vanhoucke, V, and Alemi, A, editors. Inception-v4, inception-resnet and the impact of residual connections on learning. In: Proceedings of the AAAI conference on artificial intelligence; (2017), 31.

Google Scholar

14. Prokop, M, Shin, HO, Schanz, A, and Schaefer-Prokop, CM. Use of maximum intensity projections in CT angiography: a basic review. Radiographics. (1997) 17:433–51. doi: 10.1148/radiographics.17.2.9084083

PubMed Abstract | Crossref Full Text | Google Scholar

15. Zhang, Z, Wu, C, Coleman, S, and Kerr, D. DENSE-INception U-net for medical image segmentation. Comput Methods Prog Biomed. (2020) 192:105395. doi: 10.1016/j.cmpb.2020.105395

PubMed Abstract | Crossref Full Text | Google Scholar

16. Tan, M, and Le, Q, editors. Efficientnetv2: smaller models and faster training. In: International Conference on Machine Learning; (2021): PMLR.

Google Scholar

17. Buslaev, A, Iglovikov, VI, Khvedchenya, E, Parinov, A, Druzhinin, M, and Kalinin, AA. Albumentations: fast and flexible image augmentations. Information. (2020) 11:125. doi: 10.3390/info11020125

Crossref Full Text | Google Scholar

18. Schisterman, EF, Perkins, NJ, Liu, A, and Bondell, H. Optimal cut-point and its corresponding Youden index to discriminate individuals using pooled blood samples. Epidemiology. (2005) 16:73–81. doi: 10.1097/01.ede.0000147512.81966.ba

Crossref Full Text | Google Scholar

19. DeLong, ER, DeLong, DM, and Clarke-Pearson, DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. (1988) 44:837. doi: 10.2307/2531595

Crossref Full Text | Google Scholar

20. Matsoukas, S, Morey, J, Lock, G, Chada, D, Shigematsu, T, Marayati, NF, et al. AI software detection of large vessel occlusion stroke on CT angiography: a real-world prospective diagnostic test accuracy study. J Neurointervention Surg. (2023) 15:52–6. doi: 10.1136/neurintsurg-2021-018391

PubMed Abstract | Crossref Full Text | Google Scholar

21. Czap, AL, Bahr-Hosseini, M, Singh, N, Yamal, J-M, Nour, M, Parker, S, et al. Machine learning automated detection of large vessel occlusion from mobile stroke unit computed tomography angiography. Stroke. (2022) 53:1651–6. doi: 10.1161/STROKEAHA.121.036091

PubMed Abstract | Crossref Full Text | Google Scholar

22. Meng, S, Tran, TML, Hu, M, Wang, P, Yi, T, Zhong, Z, et al. End-to-end artificial intelligence platform for the management of large vessel occlusions: a preliminary study. J Stroke Cerebrovasc Dis. (2022) 31:106753. doi: 10.1016/j.jstrokecerebrovasdis.2022.106753

PubMed Abstract | Crossref Full Text | Google Scholar

23. Olive-Gadea, M, Crespo, C, Granes, C, Hernandez-Perez, M, Pérez de la Ossa, N, Laredo, C, et al. Deep learning based software to identify large vessel occlusion on noncontrast computed tomography. Stroke. (2020) 51:3133–7. doi: 10.1161/STROKEAHA.120.030326

PubMed Abstract | Crossref Full Text | Google Scholar

24. Bensoussan, S, Finitsis, SN, Lapergue, B, Marnat, G, Sibon, I, Moulin, S, et al. Endovascular treatment of distal-M2 segment occlusions: a clinical registry and Meta-analysis. J Stroke. (2023) 25:299–302. doi: 10.5853/jos.2022.03692

PubMed Abstract | Crossref Full Text | Google Scholar

25. Fasen, B, Heijboer, R, Hulsmans, F-J, and Kwee, R. CT angiography in evaluating large-vessel occlusion in acute anterior circulation ischemic stroke: factors associated with diagnostic error in clinical practice. Am J Neuroradiol. (2020) 41:607–11. doi: 10.3174/ajnr.A6469

Crossref Full Text | Google Scholar

26. García-Tornel, Á, Ciolli, L, Rubiera, M, Requena, M, Muchada, M, Pagola, J, et al. Leptomeningeal collateral flow modifies endovascular treatment efficacy on large-vessel occlusion strokes. Stroke. (2021) 52:299–303. doi: 10.1161/STROKEAHA.120.031338

PubMed Abstract | Crossref Full Text | Google Scholar

27. Sanossian, N, Liebeskind, DS, Eckstein, M, Starkman, S, Stratton, S, Pratt, FD, et al. Routing ambulances to designated centers increases access to stroke center care and enrollment in prehospital research. Stroke. (2015) 46:2886–90. doi: 10.1161/STROKEAHA.115.010264

PubMed Abstract | Crossref Full Text | Google Scholar

28. Furlan, AJ . Time is brain. Am Heart Assoc. (2006) 37:2863–4. doi: 10.1161/01.STR.0000251852.07152.63

Crossref Full Text | Google Scholar

29. Kim, JS, and Bonovich, D. Research on intracranial atherosclerosis from the east and west: why are the results different? J Stroke. (2014) 16:105–13. doi: 10.5853/jos.2014.16.3.105

PubMed Abstract | Crossref Full Text | Google Scholar

30. Maas, MB, Lev, MH, Ay, H, Singhal, AB, Greer, DM, Smith, WS, et al. Collateral vessels on CT angiography predict outcome in acute ischemic stroke. Stroke. (2009) 40:3001–5. doi: 10.1161/STROKEAHA.109.552513

PubMed Abstract | Crossref Full Text | Google Scholar