Deep learning-assisted diagnosis of large vessel occlusion in acute ischemic stroke based on four-dimensional computed tomography angiography

Purpose: To develop deep learning models based on four-dimensional computed tomography angiography (4D-CTA) images for automatic detection of large vessel occlusion (LVO) in the anterior circulation that cause acute ischemic stroke.

Methods: This retrospective study included 104 LVO patients and 105 non-LVO patients for deep learning models development. Another 30 LVO patients and 31 non-LVO patients formed the time-independent validation set. Four phases of 4D-CTA (arterial phase P1, arterial–venous phase P2, venous phase P3 and late venous phase P4) were arranged and combined and two input methods was used: combined input and superimposed input. Totally 26 models were constructed using a modified HRNet network. Assessment metrics included the areas under the curve (AUC), accuracy, sensitivity, specificity and F1 score. Kappa analysis was performed to assess inter-rater agreement between the best model and radiologists of different seniority.

Results: The P1 + P2 model (combined input) had the best diagnostic performance. In the internal validation set, the AUC was 0.975 (95%CI: 0.878–0.999), accuracy was 0.911, sensitivity was 0.889, specificity was 0.944, and the F1 score was 0.909. In the time-independent validation set, the model demonstrated consistently high performance with an AUC of 0.942 (95%CI: 0.851–0.986), accuracy of 0.902, sensitivity of 0.867, specificity of 0.935, and an F1 score of 0.901. The best model showed strong consistency with the diagnostic efficacy of three radiologists of different seniority (k = 0.84, 0.80, 0.70, respectively).

Conclusion: The deep learning model, using combined arterial and arterial–venous phase, was highly effective in detecting LVO, alerting radiologists to speed up the diagnosis.

Introduction

Large vessel occlusion (LVO) resulting in acute ischemic stroke (AIS) is the leading cause of severe death and disability (Malhotra et al., 2017). Timely treatment can rescue salvageable brain tissue, prevent increased neurological loss and improve the long-term prognosis of AIS patients. Endovascular therapy has been proven effective in AIS patients with LVO (Campbell et al., 2015; Goyal et al., 2016), but its effectiveness is highly time-dependent (Meretoja et al., 2017). Therefore, it is essential to accurately and quickly identify LVO.

CT angiography (CTA) is commonly used for detecting LVO (Almekhlafi et al., 2019; Mayer et al., 2020). Currently, four-dimensional computed tomography angiography (4D-CTA) is an emerging technique that combines the non-invasive feature of CTA with the dynamic acquisition of DSA (Kortman et al., 2015), producing a time-resolved cerebral vascular diagram of the brain vessels from the base of the skull to the vertex (Menon et al., 2013). It displays dynamic changes of blood flow in intracranial vessels and helps detect LVO more sensitively and accurately (Wagemans et al., 2017). With optimized stroke management, the number of CTA examinations for suspected AIS is increasing, as is the workload of radiologists, which may result in a delayed diagnosis of LVO. Thus, rapidly and accurately detecting LVO on numerous CTAs suspected of having cerebrovascular problems is an urgent task.

Deep learning (DL) is a subset of machine learning which is a category of methods of artificial intelligence. By iteratively adjusting the weight layers in a deep neural network structure to transform input information into multiple levels of abstraction, DL automatically learns discriminative features and representations from data (Shen et al., 2017; Chan et al., 2020). DL has proven to be particularly useful for medical image segmentation and reconstruction, as well as for disease diagnosis and prediction. Several studies have shown that DL models have comparable or better diagnostic capabilities than human radiologists (Choi et al., 2021; Fujima et al., 2021), particularly in neuroradiology (Zaharchuk et al., 2018; Dai et al., 2020; Narayana et al., 2020).

We hypothesized that DL algorithms can effectively learn the information provided by dynamic CTA to detect LVO, thereby accelerating LVO diagnosis and ensuring timely treatment for patients. The purpose of this study was to develop and validate DL algorithm for LVO detection based on 4D-CTA, and to compare its diagnostic performance with that of radiologists of varying levels of seniority.

Materials and methods

Study participants

This retrospective study was approved by the institutional review board and ethics committee of the First Affiliated Hospital of Chongqing Medical University. Patients newly diagnosed with LVO from July 2020 to November 2021 in the Department of Neurology of our hospital were recruited. The inclusion criteria were: (1) age ≥ 18 years old, (2) diagnosis of anterior circulation LVO, (3) the “one-stop-shop” 4D-CTA + CTP examination. Patients with incomplete clinical data and poor-quality CTA scans were excluded. We also collected age-sex-matched non-LVO patients with AIS symptoms who underwent AIS protocol imaging but were negative for vessel occlusion as a control group during the same period.

A total of 134 LVO patients and 136 non-LVO patients were enrolled. Among these, 104 LVO patients and 105 non-LVO patients (July 2020 to August 2021) were included to construct the training set for DL model training, testing and internal validation at a ratio of 6:2:2. To validate the generalizability of the DL model, 30 LVO patients and 31 non-LVO patients (September 2021 to November 2021) formed the time-independent validation set.

Imaging protocols

4D-CTA images were obtained from a 320-row detector CT scanner (Aquilion ONE, Canon Medical Systems Corporation, Otawara, Japan). The scanning parameters were: 80 kV, 150–310 mA, coverage of 140–160 mm, 512 × 512 matrix, reconstruction with adaptive iterative dose reduction, 1.0 mm slice thickness, and 1.0 mm interval. The scanner provides whole-brain perfusion and dynamic vasculature information in one single examination with a single rotation of the gantry. The 4D-CTA acquisition protocol performs 19 volumetric scans by using a whole brain dynamic volume intermittent mode in a total of 60s. It used a tube current boost in the arterial enhancement peak sequence. The first volume, acquired at 310 mA, served as a mask. Subsequently, Three-volume scans at 150 mA, six-volume scans at 300 mA, and four-volume scans at 150 mA were performed sequentially per 2 s, constituting the arterial phase (11–36 s). The last five-volume scans were collected every 5 s at 150 mA for the venous phase (40–60 s). Iodine 400 (Iopamidol 400, Bracco Sine, Italy) was injected intravenously with the high-pressure syringe. The P3T technique of high-pressure syringe system (MEDRAD Stellant CT Injection System, Bayer Medical Care, Pittsburgh, United States) automatically calculate the amount and rate of contrast agents based on the patient’s gender, weight, height, and contrast agent concentration, providing a personalized contrast injection regimen.

All 19 volumetric scans were imported into a post-processing workstation (Vitrea, fX,1.0, Canon Medical Systems Corporation, Japan) and time-density curve (TDC) was generated by automatically labeling the inflow artery and outflow vein. According to the TDC, the point at which the arterial curve peaked was defined as the arterial phase (P1). The time point when the arterial curve and the venous curve intersected was defined as the arterial–venous phase (P2), the time of the venous curve to peak as the venous phase (P3), and the first time point when the venous curve entered a plateau was defined as the late venous phase (P4; Wang et al., 2022).

Definition of LVO

LVO was defined as the presence of a contrast filling defect in specific segments, including the intracranial internal carotid artery (ICA) from the clinoid segment to its terminus, M1/M2 segments of the middle cerebral artery (MCA; Luijten et al., 2022). Occlusions in isolated extracranial ICA, A1/A2, M3/M4 and posterior circulation were not included. The ground truth of LVO was established by a senior neuroradiologist (Y.M.L. with 26 years of experience), considering the patient’s history and all available imaging data.

Image preprocessing and augmentation

The dataset construction process was shown in Figure 1. Bone removal data for the four phases (P1, P2, P3, and P4) were obtained by subtracting the images of the four defined phases from the mask images. Intracranial vascular MIP map for each phase was then obtained using the maximum intensity projection (MIP) reconstruction. The MIP algorithm was implemented by Python (3.8.0), mainly using Opencv library (4.5.3) and PIL library (8.0.1).

Figure 1

Figure 1. Data preprocessing pipeline (top), overall architecture of proposed model (left), and CBAM Attention architecture (right). Data preprocessing pipeline: The intracranial vascular MIP maps were obtained based on the four phases selected by 4D-CTA. The input format was divided into single-phase input and multi-phase input (where multi-phase input has two input methods: combination or superimposition). Overall architecture: the images obtained from MIP reconstruction were fed into the modified HRNet network to extract image features, then the extracted feature maps passed through the global average pooling layer and the fully connected layer. Finally, softmax was used to generate the predicted probability of each category as output. CBAM Attention improves the classification performance of the network by weighting the extracted features in space and channels.

A total of 26 DL models were developed. Initially, four models were created separately for each phase (P1, P2, P3, and P4). Then, 11 combinations were formed based on the different combinations of four time phases. And each was trained separately using two different input methods. Input method 1—combination of different phase MIP maps, including P1 + P2, P1 + P3, P1 + P4, P2 + P3, P2 + P4, P3 + P4, P1 + P2 + P3, P1 + P2 + P4, P1 + P3 + P4, P2 + P3 + P4, and P1 + P2 + P3 + P4. Input method 2—superimposition of different phase MIP maps (i.e., re-projection of MIP maps from multiple phases into one using MIP reconstruction), including MIP (P1, P2), MIP (P1, P3), MIP (P1, P4), MIP (P2, P3), MIP (P2, P4), MIP (P3, P4), MIP (P1, P2, P3), MIP (P1, P2, P4), MIP (P1, P3, P4), MIP (P2, P3, P4), MIP (P1, P2, P3, P4). Refer to Figure 2 for details.

Figure 2

Figure 2. Three types of input to the model. Single phase (top): Each phase was input to the network model individually and the classification results were obtained. Multi-phase Way 1 (middle): Combination of multiple phase images for input. The output predicted probabilities were averaged to obtain the final model results. Multi-phase Way 2 (bottom): Superimposition of multiple phase images for input. The MIP images from multiple phases were again subjected to MIP processing to obtain a single superimposed image, which was input to obtain the final model classification results.

Conventional data enhancement techniques such as online rotation, flipping, cropping, and luminance changes were applied to enhance the learning representation of the model. These operations were implemented in Python (version 3.8.0).

DL model development

We used the HRNet network to build DL model for LVO detection due to its superiority in image classification and segmentation (Jansen et al., 2018; Smith et al., 2018). To improve its performance, we made three modifications to the original HRNet network (Figure 1), It was mainly reflected in: (1) reduced the network structure from 4 to 3 stages (each yellow area represents one stage), (2) improved the successive convolution operations in each stage (modified to Modified Unit), (3) introduced channel and spatial attention mechanisms to the Modified Unit to enhance the network’s learning ability. In short, the network workflow involved feeding MIP-reconstructed images into the modified HRNet network to extract image features, then passed the feature maps through the global average pooling layer and fully connected layer, and finally generated the predicted probability of each category as output using softmax.

All models were implemented on the PyTorch library with NVIDIA GPU (GeForce GTX 2070 SUPER). The input image size was set to 256 × 256, and the batch size was set as 16. ADAM optimizer was used with an initial learning rate of 0.0001. The momentum and training epoch was set to 0.9 and 200, respectively. The warm-up strategy was applied in the first five epochs, and the learning rate was decayed at the 80th and 150th epochs with a decay ratio of 0.1.

Radiologist performance

To evaluate the proposed DL model’s clinical applicability, we compared the diagnostic performance of the best model with that of radiologists with varying expertise. Three radiologists, Reader 1 (J.L with 8 years of experience), Reader 2 (L.Q.D with 5 years of experience), and Reader 3, a trainee radiologist (S.R.G, a trainee radiologist with 1 year of experience), identified LVO or non-LVO from 4D-CTA images of the temporal validation set. They were blinded to clinical information, radiologic reports, and other examination findings.

Statistical analysis

The normality of data distribution was tested, with continuous variables reported as mean ± standard deviations (SD) or medians and interquartile ranges (IQRs), and categorical variables reported as proportions. Continuous variables were analyzed using Student’s t-test or Mann–Whitney U-test. The performance of the proposed DL models was evaluated by calculating the areas under the curve (AUC) using receiver operating characteristic curve (ROC) analysis for the training and temporal validation sets. We also compared performance using accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and F1 score. Differences in ROC among the DL models were evaluated using the Delong test. Inter-rater agreement between the best model and each reader was assessed using the kappa analysis. Statistical significance was considered at p < 0.05. Statistical analysis was performed using SPSS (version 26.0, IBM, New York) and MedCalc software (version 20.115, https://www.medcalc.org/).

Results

Patient demographic data, clinical characteristics

Table 1 summarized the demographic characteristics of 270 subjects, including 134 LVO patients (83 males and 51 females) and 136 non-LVO patients (83 males and 53 females). The training set comprised 104 LVO patients (mean age, 65.88 ± 12.82 years; 67 men) and 105 non-LVO patients (mean age, 62.85 ± 10.77 years; 64 men). Of these, the LVO subgroup contained 63 M1 (60.58%), 19 M2 (18.27%), 14 ICA (13.46%), and 8 tandem (7.69%). The temporal validation set consisted of 30 LVO patients (mean age, 69.50 ± 9.89 years; 16 men) and 31 non-LVO patients (mean age, 65.51 ± 12.57 years; 19 men), with 19 M1 (63.33%), 6 M2 (20%), 2 ICA (6.67%), and 3 tandem (10%) in the LVO subgroup. Both sets showed that LVO patients had higher NIHSS median scores than non-LVO patients [9 (2–15) vs. 2 (1–4), p < 0.001; 11 (2–18) vs. 0 (0–1), p < 0.001]. In the overall cohort, a NIHSS score cut-off of 6.5 was identified as the optimal threshold for detecting LVO, yielding an AUC of 0.853 (95% CI 0.804–0.903). This NIHSS cut-off demonstrated a sensitivity of 69% and specificity of 96%.

Table 1

Table 1. Demographics in the training and temporal validation cohort.

Performance of the DL models

The DL models developed in four separate phases had AUCs above 0.700 (Table 2). Of these, model P1 and P2 showed superior performance with AUCs above 0.800. The DL model trained on P1 alone achieved an AUC of 0.844 (95%CI: 0.704–0.935), with accuracy of 0.867, sensitivity of 0.963, specificity of 0.722 and F1 score of 0.855. The DL model trained on P2 alone achieved an AUC of 0.897 (95%CI: 0.770–0.968), slightly higher than P1, with accuracy of 0.844, sensitivity of 0.889, specificity of 0.778 and F1 score of 0.836.

Table 2

Table 2. The performance of the DL model based on four separate phases.

The results of DL models using combinations of MIP maps from different phases as input way were shown in Table 3. All models had AUCs above 0.800 for the Internal validation set. Among them, the P1 + P2 model had superior diagnostic performance with an increased AUC of 0.975 (95%CI: 0.878–0.999) compared to model P1, and achieved an accuracy of 0.911, sensitivity of 0.889, specificity of 0.944, and an F1 score of 0.909. Consistently high diagnostic efficacy for the P1 + P2 model was observed in the temporal validation set, with an AUC of 0.942 (95% CI: 0.851–0.986), accuracy of 0.902, sensitivity of 0.867 and specificity of 0.935, and F1 score of 0.901. This was followed by the P1 + P2 + P3 model with an AUC of 0.916 (95%CI: 0.794–0.978), accuracy of 0.867, sensitivity of 0.852, specificity of 0.889, and F1 score of 0.863. However, the Delong test showed that the improvement was not statistically significant (Supplementary Table S1).

Table 3

Table 3. The performance metrics for DL models with combinations of MIP maps at different stages as input way.

Table 4 showed results of DL models using superimposition of MIP maps from different phases as the input method. The average AUC was 0.798, significantly lower than the mean AUC of 0.863 for the combination input models. The DL models trained on MIP (P1, P2, P3) and MIP (P1, P2) showed relatively good efficacy in identifying LVO, with AUC of 0.879 (95%CI: 0.747–0.957) and 0.860 (95%CI: 0.724–0.945), respectively. However, both models were less effective than the best model P1 + P2. ROC results for the top two high-performing DL models with different input methods were shown in Figure 3. Overall, we constructed 26 models, and the DL model trained on P1 + P2 using the combined input approach had the highest diagnostic efficacy, surpassing the models composed by the mono-phase and superimposed input approaches.

Table 4

Table 4. The performance of DL models using superimposition of MIP maps from different phases as input way.

Figure 3

Figure 3. The receiver operating characteristic curves results (ROC) for the top two high-performing DL models which were, respectively, built by single-phase input, multi-phases combined input and multi-phases superimposed input way. (A) Results of ROC analysis on the internal validation cohort. (B) Results of ROC analysis on the temporal validation cohort.

Discrimination performance of the best model and radiologists with different levels of expertise

Based on the time-independent validation set, we compared the diagnostic efficacy of the best DL model with that of radiologists with different levels of experiences (Figure 4; Supplementary Table S2). The best DL model outperformed Reader 3 for accuracy (0.902, 0.885, respectively), sensitivity (0.867, 0.833, respectively), specificity (0.935, 0.935, respectively), PPV (0.929, 0.926, respectively), NPV (0.879, 0.853, respectively), and F1 score (0.901, 0.885, respectively). Moreover, the performance of the best DL model was almost identical to Reader 1 (k = 0.84) and had strong inter-rater agreement with Reader 2 and Reader 3 (k = 0.80, 0.70, respectively).

Figure 4

Figure 4. The confusion matrix of the best DL model (A) and the radiologists with different experiences (B–D) in the temporal validation cohort.

We evaluated the diagnostic accuracy of the top-performing DL model and radiologists with varying levels of experience (Figure 4; Supplementary Table S2) using a time-independent validation set. Compared to Reader 3, the DL model demonstrated superior performance across all measures, including accuracy (0.902 vs. 0.885), sensitivity (0.867 vs. 0.833), specificity (0.935 vs. 0.935), PPV (0.929 vs. 0.926), NPV (0.879 vs. 0.853), and F1 score (0.901 vs. 0.885). The DL model also showed strong agreement with Reader 1 (k = 0.84) and had high inter-rater agreement with Reader 2 and Reader 3 (k = 0.80 and 0.70, respectively).

Discussion

In this study, we developed DL models based on dynamic 4D-CTA to detect LVO and investigated the efficacy of two different input methods, namely combining and superimposing multi-phase MIP images. Our results showed that the DL algorithm had high efficacy in LVO detection, with the P1 + P2 model in the combination input approach having the highest detection efficacy. The combination of arterial and arterial–venous phase images provided richer temporal information, enabling more accurate and timely detection of LVO.

The importance of timely treatment in managing AIS is underscored by the concept of “time is brain,” emphasizing the critical role of rapid reperfusion therapy in salvaging at-risk brain tissue and improving patient outcomes (Goyal et al., 2016; Jansen et al., 2018). Society of Neurointerventional Surgery suggest an ideal time for CTA interpretation < 10 min (Smith et al., 2018). However, challenges in the emergency setting, such as the large number of patients and the shortage of neuroradiologists, may hinder meeting this time target, resulting in delays in evaluation for LVO patients. DL-based algorithms provide a solution to simplify the diagnostic process by automatically identifying LVO, thereby ensuring timely access to diagnostic imaging and intervention. Based on the strong performance of HRNet in image classification and segmentation (Ding et al., 2021; Li et al., 2021), we improved the original HRNet network for constructing our LVO DL-model The improved HRNet network excels at extracting richer semantic features, resulting in improved performance and increased utility for real-world applications.

DL has shown promise in developing models and software for diagnosing LVO (Rava et al., 2021; Yahav-Dovrat et al., 2021; Cimflova et al., 2022; Czap et al., 2022; Seker et al., 2022), mostly based on monophasic CTA. However, 4D-CTA provides dynamic multi-phase scanning, offering temporal resolution for a comprehensive assessment of hemodynamic changes in AIS patients (Frölich et al., 2014; Kortman et al., 2015). Currently, only a few DL studies have been based on 4D CTA for LVO detection. Meijs et al. (2020) used nTTS (time to signal) maps for LVO detection, while Bathla et al. (2022) employed 5 time series data for LVO detection and localization. The results of these studies demonstrated the feasibility of DL models based on 4D CTA for LVO detection, yet they did not investigate the impact of different phases.

Without increasing the radiation dose, we obtained four-phase images of arterial, arterial–venous, venous, and late venous using TDC, ensuring scientific accuracy and precision. Our results found that arterial–venous phase was comparable to arterial phase for LVO assessment, and the combination of arterial–venous and arterial phase improved performance to a greater extent, compared to the arterial phase alone. Additionally, combining the late venous phase with the arterial phase also improved LVO detection effectiveness to some extent, consistent with the results of Stib et al. (2020), who reported that the delayed phase improved the diagnostic performance of DL models. These findings suggested that vascular flow changes and pathophysiological alterations in various periods aid in LVO diagnosis, including the disappearance or invisibility of early occluded arteries, delayed enhancement of occluded arteries and collateral vessel filling (Frölich et al., 2012). However, there was no statistically significant difference in the improvement of the AUC of the multiphase joint model in our study, showing only a trend, which may be related to the relatively limited sample size.

Moreover, we investigated two input methods for DL model development: combining and superimposing MIP maps of different phases. Our results showed that the combined input method outperformed the superimposed input method, which may be attributed to the MIP technique. 4D-CTA highlights blood flow in different vessels at different phases (reflected as different brightness of the image), while MIP projects the maximum value in the same location and sequence onto the final image. With the combined input method, by combining and stitching separate MIP images from each period, vascular changes can be observed more clearly. Conversely, the superimposed input method projects highlighted areas from each stage onto the final image, leading to reduced image quality and negatively impacting the final performance of the DL model. It underscoring the importance of selecting appropriate input methods in DL model development.

Currently, although all CTAs are reviewed by radiologists, there is no guarantee that LVO will be prioritized among the many suspected AIS cases. The comparative analysis between the best DL model and radiologists with varying levels of expertise revealed good consistency. The best DL model demonstrated a relatively higher accuracy and sensitivity, comparable specificity to less experienced trainee radiologist. These findings further emphasize the potential of DL models as valuable diagnostic support tools in prioritizing LVO detection and facilitating timely treatment decisions. It can flag the presence of LVO in unreviewed reports, alerting radiologists for prioritized review and preventing delayed diagnosis or omission.

Additionally, NIHSS scores showed a significant difference between the LVO and non-LVO groups, highlighting its role as a major clinical predictor of LVO (Fischer et al., 2005; Heldner et al., 2013) and its current utility in prehospital EVT decision-making (Václavík et al., 2018; Cabal et al., 2021). However, NIHSS assessments are relatively time-consuming and dependent on experienced neurologists. In contrast, the best DL model outperformed the NIHSS by visually detecting occluded vessels using comprehensive information from CTA images, offering more effective guidance for treatment decisions.

Our study had several limitations. Firstly, this was a single-center study with a relatively small sample size and CTA images obtained from the same scanner with consistent acquisition protocols, leading to potential sample selection bias. Future studies should consider multicenter studies with larger sample sizes and images from different acquisition parameters. Secondly, the proposed DL algorithm was not externally validated using a validation set from another institution, but a temporal validation set was used to assess its performance over time. The generalization capability of the algorithm needs to be further evaluated on multicenter external data. Thirdly, the algorithm only identifies LVO of the ICA, M1, and M2, excluding posterior circulation LVO, and does not provide information about a specific occluded segment. Therefore, we will collect more LVO cases in the anterior and posterior circulation and enhance the algorithm to better aid radiologists in detecting LVO.

Conclusion

In conclusion, our study highlights the potential of the DL model in detecting LVO in AIS patients using 4D-CTA. Through the utilization of advanced image processing techniques and comprehensive temporal information, DL models can accurately and rapidly identify LVO, especially when using the combination of arteriovenous phase images and arterial phase images, which demonstrated the highest detection efficacy. This helps to accelerate the screening and diagnosis of LVO in urgent clinical settings, ultimately improving patient prognosis.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving humans were approved by the First Affiliated Hospital of Chongqing Medical University. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

YP: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Writing – original draft, Writing – review & editing. JLiu: Data curation, Investigation, Validation, Writing – review & editing. RY: Formal analysis, Methodology, Software, Visualization, Writing – review & editing. JWu: Data curation, Investigation, Writing – review & editing. JLi: Methodology, Writing – review & editing. LD: Validation, Writing – review & editing. SG: Data curation, Validation, Writing – review & editing. YY: Data curation, Validation, Writing – review & editing. YL: Supervision, Writing – review & editing, Resources. SC: Project administration, Resources, Software, Supervision, Writing – review & editing. JWa: Conceptualization, Methodology, Project administration, Resources, Supervision, Writing – review & editing.

Funding

The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnins.2024.1329718/full#supplementary-material

Abbreviations

AIS, Acute ischemic stroke; LVO, Large vessel occlusion; CTA, Computed tomography angiography; 4D-CTA, Four-dimensional computed tomography angiography; DL, Deep learning; ICA, Internal carotid artery; MCA, Middle cerebral artery; MIP, Maximum intensity projection; AUC, Areas under the curve; ROC, Receiver operating characteristic curve; PPV, Positive predictive value; NPV, Negative predictive value; NIHSS, National institutes of health stroke scale baseline score.

References

Almekhlafi, M. A., Kunz, W. G., Menon, B. K., McTaggart, R. A., Jayaraman, M. V., Baxter, B. W., et al. (2019). Imaging of patients with suspected large-vessel occlusion at primary stroke centers: available modalities and a suggested approach. Am. J. Neuroradiol. 40, 396–400. doi: 10.3174/ajnr.A5971

PubMed Abstract | Crossref Full Text | Google Scholar

Bathla, G., Durjoy, D., Priya, S., Samaniego, E., and Derdeyn, C. P. (2022). Image level detection of large vessel occlusion on 4D-CTA perfusion data using deep learning in acute stroke. J. Stroke Cerebrovasc. Dis. 31:106757:106757. doi: 10.1016/j.jstrokecerebrovasdis.2022.106757

PubMed Abstract | Crossref Full Text | Google Scholar

Cabal, M., Machova, L., Vaclavik, D., Jasso, P., Holes, D., Volny, O., et al. (2021). New prehospital triage for stroke patients significantly reduces transport time of EVT patients without delaying IVT. Front. Neurol. 12:676126. doi: 10.3389/fneur.2021.676126

PubMed Abstract | Crossref Full Text | Google Scholar

Campbell, B. C. V., Donnan, G. A., Lees, K. R., Hacke, W., Khatri, P., Hill, M. D., et al. (2015). Endovascular stent thrombectomy: the new standard of care for large vessel ischaemic stroke. Lancet Neurol. 14, 846–854. doi: 10.1016/S1474-4422(15)00140-4

PubMed Abstract | Crossref Full Text | Google Scholar

Chan, H. P., Hadjiiski, L. M., and Samala, R. K. (2020). Computer-aided diagnosis in the era of deep learning. Med. Phys. 47, E218–E227. doi: 10.1002/mp.13764

Crossref Full Text | Google Scholar

Choi, H., Kim, H., Hong, W., Park, J., Hwang, E. J., Park, C. M., et al. (2021). Prediction of visceral pleural invasion in lung cancer on CT: deep learning model achieves a radiologist-level performance with adaptive sensitivity and specificity to clinical needs. Eur. Radiol. 31, 2866–2876. doi: 10.1007/s00330-020-07431-2

PubMed Abstract | Crossref Full Text | Google Scholar

Cimflova, P., Golan, R., Ospel, J. M., Sojoudi, A., Duszynski, C., Elebute, I., et al. (2022). Validation of a machine learning software tool for automated large vessel occlusion detection in patients with suspected acute stroke. Neuroradiology 64, 2245–2255. doi: 10.1007/s00234-022-02978-x

PubMed Abstract | Crossref Full Text | Google Scholar

Czap, A. L., Bahr-Hosseini, M., Singh, N., Yamal, J. M., Nour, M., Parker, S., et al. (2022). Machine learning automated detection of large vessel occlusion from Mobile stroke unit computed tomography angiography. Stroke 53, 1651–1656. doi: 10.1161/STROKEAHA.121.036091

PubMed Abstract | Crossref Full Text | Google Scholar

Dai, X., Huang, L., Qian, Y., Xia, S., Chong, W., Liu, J., et al. (2020). Deep learning for automated cerebral aneurysm detection on computed tomography images. Int. J. Comput. Assist. Radiol. Surg. 15, 715–723. doi: 10.1007/s11548-020-02121-2

PubMed Abstract | Crossref Full Text | Google Scholar

Ding, M., Kang, Y., Yuan, Z., Shan, X., and Cai, Z. (2021). Detection of facial landmarks by a convolutional neural network in patients with oral and maxillofacial disease. Int. J. Oral Maxillofac. Surg. 50, 1443–1449. doi: 10.1016/j.ijom.2021.01.002

PubMed Abstract | Crossref Full Text | Google Scholar

Fischer, U., Arnold, M., Nedeltchev, K., Brekenfeld, C., Ballinari, P., Remonda, L., et al. (2005). NIHSS score and arteriographic findings in acute ischemic stroke. Stroke 36, 2121–2125. doi: 10.1161/01.STR.0000182099.04994.fc

PubMed Abstract | Crossref Full Text | Google Scholar

Frölich, A. M. J., Psychogios, M. N., Klotz, E., Schramm, R., Knauth, M., and Schramm, P. (2012). Antegrade flow across incomplete vessel occlusions can be distinguished from retrograde collateral flow using 4-dimensional computed tomographic angiography. Stroke 43, 2974–2979. doi: 10.1161/STROKEAHA.112.668889

PubMed Abstract | Crossref Full Text | Google Scholar

Frölich, A. M. J., Wolff, S. L., Psychogios, M. N., Klotz, E., Schramm, R., Wasser, K., et al. (2014). Time-resolved assessment of collateral flow using 4D CT angiography in large-vessel occlusion stroke. Eur. Radiol. 24, 390–396. doi: 10.1007/s00330-013-3024-6

PubMed Abstract | Crossref Full Text | Google Scholar

Fujima, N., Andreu-Arasa, V. C., Onoue, K., Weber, P. C., Hubbell, R. D., Setty, B. N., et al. (2021). Utility of deep learning for the diagnosis of otosclerosis on temporal bone CT. Eur. Radiol. 31, 5206–5211. doi: 10.1007/s00330-020-07568-0

PubMed Abstract | Crossref Full Text | Google Scholar

Goyal, M., Menon, B. K., van Zwam, W., Dippel, D. W., Mitchell, P. J., Demchuk, A. M., et al. (2016). Endovascular thrombectomy after large-vessel ischaemic stroke: a meta-analysis of individual patient data from five randomised trials. Lancet 387, 1723–1731. doi: 10.1016/S0140-6736(16)00163-X

PubMed Abstract | Crossref Full Text | Google Scholar

Heldner, M. R., Zubler, C., Mattle, H. P., Schroth, G., Weck, A., Mono, M. L., et al. (2013). National Institutes of Health stroke scale score and vessel occlusion in 2152 patients with acute ischemic stroke. Stroke 44:1153. doi: 10.1161/STROKEAHA.111.000604

Crossref Full Text | Google Scholar

Jansen, I. G. H., Mulder, M. J. H. L., and Goldhoorn, R. B. MR CLEAN Registry investigators (2018). Endovascular treatment for acute ischaemic stroke in routine clinical practice: prospective, observational cohort study (MR CLEAN registry). BMJ 360:k949. doi: 10.1136/bmj.k949

PubMed Abstract | Crossref Full Text | Google Scholar

Kortman, H. G. J., Smit, E. J., Oei, M. T. H., Manniesing, R., Prokop, M., and Meijer, F. J. A. (2015). 4D-CTA in neurovascular disease: A review. Am. J. Neuroradiol. 36, 1026–1033. doi: 10.3174/ajnr.A4162

PubMed Abstract | Crossref Full Text | Google Scholar

Li, Y. S., Cui, J., Sheng, Y., Liang, X., Wang, J., Chang, E. I. C., et al. (2021). Whole brain segmentation with full volume neural network. Comput. Med. Imaging Graph. 93:101991. doi: 10.1016/j.compmedimag.2021.101991

PubMed Abstract | Crossref Full Text | Google Scholar

Luijten, S. P. R., Wolff, L., Duvekot, M. H. C., van Doormaal, P., Moudrous, W., Kerkhoff, H., et al. (2022). Diagnostic performance of an algorithm for automated large vessel occlusion detection on CT angiography. J Neurointervent Surg 14, 794–798. doi: 10.1136/neurintsurg-2021-017842

PubMed Abstract | Crossref Full Text | Google Scholar

Malhotra, K., Gornbein, J., and Saver, J. L. (2017). Ischemic strokes due to large-vessel occlusions contribute disproportionately to stroke-related dependence and death: A review. Front. Neurol. 8:651. doi: 10.3389/fneur.2017.00651

Crossref Full Text | Google Scholar

Mayer, S. A., Viarasilpa, T., Panyavachiraporn, N., Brady, M., Scozzari, D., van Harn, M., et al. (2020). CTA-for-all impact of emergency computed tomographic angiography for all patients with stroke presenting within 24 hours of onset. Stroke 51, 331–334. doi: 10.1161/STROKEAHA.119.027356

PubMed Abstract | Crossref Full Text | Google Scholar

Meijs, M., Meijer, F. J. A., Prokop, M., Ginneken, B., and Manniesing, R. (2020). Image-level detection of arterial occlusions in 4D-CTA of acute stroke patients using deep learning. Med. Image Anal. 66:101810:101810. doi: 10.1016/j.media.2020.101810

PubMed Abstract | Crossref Full Text | Google Scholar

Menon, B. K., O’Brien, B., Bivard, A., Spratt, N. J., Demchuk, A. M., Miteff, F., et al. (2013). Assessment of leptomeningeal collaterals using dynamic CT angiography in patients with acute ischemic stroke. J. Cereb. Blood Flow Metab. 33, 365–371. doi: 10.1038/jcbfm.2012.171

PubMed Abstract | Crossref Full Text | Google Scholar

Meretoja, A., Keshtkaran, M., Tatlisumak, T., Donnan, G. A., and Churilov, L. (2017). Endovascular therapy for ischemic stroke. Neurology 88, 2123–2127. doi: 10.1212/WNL.0000000000003981

PubMed Abstract | Crossref Full Text | Google Scholar

Narayana, P. A., Coronado, I., Sujit, S. J., Wolinsky, J. S., Lublin, F. D., and Gabr, R. E. (2020). Deep learning for predicting enhancing lesions in multiple sclerosis from noncontrast MRI. Radiology 294, 398–404. doi: 10.1148/radiol.2019191061

PubMed Abstract | Crossref Full Text | Google Scholar

Rava, R. A., Peterson, B. A., Seymour, S. E., Snyder, K. V., Mokin, M., Waqas, M., et al. (2021). Validation of an artificial intelligence-driven large vessel occlusion detection algorithm for acute ischemic stroke patients. Neuroradiol. J. 34, 408–417. doi: 10.1177/1971400921998952

PubMed Abstract | Crossref Full Text | Google Scholar

Seker, F., Pfaff, J. A. R., Mokli, Y., Berberich, A., Namias, R., Gerry, S., et al. (2022). Diagnostic accuracy of automated occlusion detection in CT angiography using e-CTA. Int. J. Stroke 17, 77–82. doi: 10.1177/1747493021992592

PubMed Abstract | Crossref Full Text | Google Scholar

Shen, D. G., Wu, G. R., and Suk, H. I. (2017). Deep learning in medical image analysis. Annu. Rev. Biomed. Eng. 19, 221–248. doi: 10.1146/annurev-bioeng-071516-044442

Crossref Full Text | Google Scholar

Smith, E. E., Kent, D. M., Bulsara, K. R., Leung, L. Y., Lichtman, J. H., Reeves, M. J., et al. (2018). Accuracy of prediction instruments for diagnosing large vessel occlusion in individuals with suspected stroke: A systematic review for the 2018 guidelines for the early Management of Patients with Acute Ischemic Stroke. Stroke 49, e111–e122. doi: 10.1161/STR.0000000000000160

PubMed Abstract | Crossref Full Text | Google Scholar

Stib, M. T., Vasquez, J., Dong, M. P., Kim, Y. H., Subzwari, S. S., Triedman, H. J., et al. (2020). Detecting large vessel occlusion at multiphase CT angiography by using a deep convolutional neural network. Radiology 297, 640–649. doi: 10.1148/radiol.2020200334

PubMed Abstract | Crossref Full Text | Google Scholar

Václavík, D., Bar, M., Klečka, L., Holeš, D., Čábal, M., and Mikulík, R. (2018). Prehospital stroke scale (FAST PLUS test) predicts patients with intracranial large vessel occlusion. Brain Behav. 8:e01087. doi: 10.1002/brb3.1087

PubMed Abstract | Crossref Full Text | Google Scholar

Wagemans, B. A., van Zwam, W. H., Nelemans, P. J., van Oostenbrugge, R. J., and Postma, A. A. (2017). 4D-CTA improves diagnostic certainty and accuracy in the detection of proximal intracranial anterior circulation occlusion in acute ischemic stroke. PLoS One 12:e0172356. doi: 10.1371/journal.pone.0172356

PubMed Abstract | Crossref Full Text | Google Scholar

Wang, J., Tan, D., Liu, J., Wu, J., Huang, F., Xiong, H., et al. (2022). Merging multiphase CTA images and training them simultaneously with a deep learning algorithm could improve the efficacy of AI models for lateral circulation assessment in ischemic stroke. Diagnostics 12:1562. doi: 10.3390/diagnostics12071562

PubMed Abstract | Crossref Full Text | Google Scholar

Yahav-Dovrat, A., Saban, M., Merhav, G., Lankri, I., Abergel, E., Eran, A., et al. (2021). Evaluation of artificial intelligence-powered identification of large-vessel occlusions in a comprehensive stroke center. AJNR Am. J. Neuroradiol. 42, 247–254. doi: 10.3174/ajnr.A6923

PubMed Abstract | Crossref Full Text | Google Scholar

Zaharchuk, G., Gong, E., Wintermark, M., Rubin, D., and Langlotz, C. P. (2018). Deep learning in neuroradiology. AJNR Am. J. Neuroradiol. 39, 1776–1784. doi: 10.3174/ajnr.A5543

PubMed Abstract | Crossref Full Text | Google Scholar