A deep learning algorithm to identify carotid plaques and assess their stability

Background: Carotid plaques are major risk factors for stroke. Carotid ultrasound can help to assess the risk and incidence rate of stroke. However, large-scale carotid artery screening is time-consuming and laborious, the diagnostic results inevitably involve the subjectivity of the diagnostician to a certain extent. Deep learning demonstrates the ability to solve the aforementioned challenges. Thus, we attempted to develop an automated algorithm to provide a more consistent and objective diagnostic method and to identify the presence and stability of carotid plaques using deep learning.

Methods: A total of 3,860 ultrasound images from 1,339 participants who underwent carotid plaque assessment between January 2021 and March 2023 at the Shanghai Eighth People’s Hospital were divided into a 4:1 ratio for training and internal testing. The external test included 1,564 ultrasound images from 674 participants who underwent carotid plaque assessment between January 2022 and May 2023 at Xinhua Hospital affiliated with Dalian University. Deep learning algorithms, based on the fusion of a bilinear convolutional neural network with a residual neural network (BCNN-ResNet), were used for modeling to detect carotid plaques and assess plaque stability. We chose AUC as the main evaluation index, along with accuracy, sensitivity, and specificity as auxiliary evaluation indices.

Results: Modeling for detecting carotid plaques involved training and internal testing on 1,291 ultrasound images, with 617 images showing plaques and 674 without plaques. The external test comprised 470 ultrasound images, including 321 images with plaques and 149 without. Modeling for assessing plaque stability involved training and internal testing on 764 ultrasound images, consisting of 494 images with unstable plaques and 270 with stable plaques. The external test was composed of 279 ultrasound images, including 197 images with unstable plaques and 82 with stable plaques. For the task of identifying the presence of carotid plaques, our model achieved an AUC of 0.989 (95% CI: 0.840, 0.998) with a sensitivity of 93.2% and a specificity of 99.21% on the internal test. On the external test, the AUC was 0.951 (95% CI: 0.962, 0.939) with a sensitivity of 95.3% and a specificity of 82.24%. For the task of identifying the stability of carotid plaques, our model achieved an AUC of 0.896 (95% CI: 0.865, 0.922) on the internal test with a sensitivity of 81.63% and a specificity of 87.27%. On the external test, the AUC was 0.854 (95% CI: 0.889, 0.830) with a sensitivity of 68.52% and a specificity of 89.49%.

Conclusion: Deep learning using BCNN-ResNet algorithms based on routine ultrasound images could be useful for detecting carotid plaques and assessing plaque instability.

1 Introduction

The incidence of stroke exceeds 100 per 100,000 and is increasing year by year. And because the death rate from stroke is close to 300 per 100,000, the high morbidity and mortality rates make the cost of stroke treatment among the highest (Hu et al., 2020). Up to one quarter of strokes can be attributed to the rupture and shedding of unstable carotid plaques (Murray et al., 2018; Sun et al., 2018; Parish et al., 2019; Kopczak et al., 2020). Plaque stability is commonly based on the following features: shape, structure, lipids, fibrous caps, and calcification (Baradaran and Gupta, 2021; Bos et al., 2021).

Traditional carotid plaque identification relies on obtaining images of patients’ necks through techniques such as ultrasound examination (Murray et al., 2018), computed tomography (CT) (Zhang et al., 2022), magnetic resonance imaging (MRI) (Hajhosseiny et al., 2019) and so on, which are then analyzed by experienced doctors for diagnosis. These imaging techniques are categorized into invasive and non-invasive methods. To reduce the burden on patients and to acquire the necessary images more rapidly, ultrasound examination, known for its non-invasive nature and capability for real-time imaging, is widely used in clinical settings (Tjoa and Guan, 2021). Due to factors like the uneven distribution of medical resources across different regions and the inherent limitations of ultrasound imaging technology, ultrasound images are prone to a significant amount of noise. The diagnostic process conducted by doctors can be influenced by variables such as the expertise of the physician, their knowledge background, fatigue, and other factors. When combined with the interference from these noises, subjective bias can occur in the diagnostic outcomes.

Before the application of deep learning to carotid plaque identification, researchers often used methods such as edge detection algorithms (Kerwin et al., 2007), region-growing algorithms (Francois et al., 2011), and texture-based analysis for plaque detection (Molinari et al., 2007). However, edge detection algorithms and region-growing algorithms are limited by the image quality and are particularly sensitive to changes in brightness and color. Texture-based analysis methods can be significantly affected by the high-density noise present in ultrasound images.

Therefore, we believe that these challenges can be solved through automated analysis through deep learning (Wang et al., 2021), which has already shown great promise in medical image analysis, spanning from screening, diagnosing to prognosis prediction in varying disease such as lung cancer, skin cancer, and breast cancer (Chaudhary et al., 2018; Roy-Cardinal et al., 2019; Ying et al., 2022; Wang et al., 2023). Machine learning could detect carotid plaques (Chen et al., 2021), but requires manual segmentation. Deep learning, in contrast, automates feature extraction for potentially greater reliability and accuracy (LeCun et al., 2015).

Among the various deep learning algorithms reported so far, Convolutional Neural Networks (CNNs) (Wang and Qi, 2023) such as VGG-16, ResNet (Johri et al., 2020) and their variants have shown particularly good performance in image feature classification. Therefore, we attempted to directly apply these networks to the task of plaque recognition and classification in ultrasound images. However, these traditional network models seem to struggle with processing conventional ultrasound images, which are often filled with noise, especially where plaques usually occupy a relatively small area. Conventionally, increasing the number of network parameters theoretically benefits the model’s fitting effect. We started testing with ResNet-18 and found that ResNet-50, which has more parameters, improved the model’s performance. Consequently, we explored using bilinear CNNs on top of Resnet to increase the network’s parameters without increasing its depth, providing a more detailed representation of ultrasound image features. This approach makes the algorithm more robust to heterogeneity and noise in images. The ability to use image features can enhance supervised information, aiding in improving the proficiency of image classification.

Therefore, the objective of this study is to develop and validate a novel BCNN-ResNet that facilitates automated detection of carotid plaques and assessment of their instability from routine ultrasound images, thereby aiding in the efficient screening and prevention of stroke associated with carotid artery disease.

2 Dataset and methods

2.1 Data and quality control

2.1.1 The BCNN-ResNet dataset

Doppler ultrasound images of the bilateral carotids, which underwent health checkup and carotid plaque screening. A total of 3,860 ultrasound images from 1,339 participants who underwent carotid plaque assessment between January 2021 and March 2023 at the Shanghai Eighth People’s Hospital were divided into a 4:1 ratio for training and internal testing. The external test included 1,564 ultrasound images from 674 participants who underwent carotid plaque assessment between January 2022 and May 2023 at Xinhua Hospital affiliated with Dalian University. Participants were excluded from the study if longitudinal ultrasound images were unavailable, annotated measurement size markers, insufficient quality. At last, Modeling for detecting carotid plaques involved training and internal testing on 1,291 ultrasound images, with 617 images showing plaques and 674 without plaques. The external test comprised 470 ultrasound images, including 321 images with plaques and 149 without. Modeling for assessing plaque stability involved training and internal testing on 764 ultrasound images, consisting of 494 images with unstable plaques and 270 with stable plaques. The external test was composed of 279 ultrasound images, including 197 images with unstable plaques and 82 with stable plaques. Details see Supplementary Figure S1.

2.1.2 Carotid plaques ultrasound imaging quality control

Carotid plaques and stability were assessed using longitudinal scanning images based on the 2020 American Society of Echocardiography guidelines (Zhou et al., 2021). The presence of plaques were defined as deposits with intima-media thickness (IMT) ≥1.5 mm or that protruded into the lumen or whose thickness exceeded 50% of the peripheral IMT. Plaques were considered stable if their morphology was regular in shape, their echogenicity was uniform, and their surface was smooth and continuous with an intact fibrous cap. Plaques were considered unstable when they exhibited inordinate morphology, uneven echogenicity, a thin fibrous cap that was incomplete or ruptured, a hypoechoic plaque or ulcerated plaque that revealed damaged areas on the plaque surface, or lipid cores that covered more than 40% of the plaque area. Color Doppler ultrasonography was conducted with a 7.5–12.0 MHz probe on patients in a supine position. Five imaging systems were used: EPIC7c (Philips, Amsterdam, Netherlands), Affiniti70 (Philips, Amsterdam, Netherlands), Aloka ARIETTA 60 (HITACHI, Tokyo, Japan), GEs8 (GE, Fairfield, Connecticut, United States), Aplio400 (TOSHIBA, Tokyo, Japan). The probe was longitudinally scanned perpendicularly to the neck in order to maintain a scan depth of 3–4 cm. The carotid was placed in the middle of the image in the case of patients diagnosed with a normal artery; otherwise, the plaque was placed in the middle of the image (show as Figure 1). During scanning, gain was 50–70 dB; power, 40–50 Hz; and the angle between the flow beam and the speed of sound, ≤60°. One or two images in both DICOM and JPG formats, without measurement size markers, were analyzed for each participants. Resolution was standardized across all images to be 256 × 256 pixels.

Figure 1

Figure 1. Schematic diagram of carotid ultrasound image retention standards. (A–D) The normal carotid. (E-H) The carotid plaques.

Using Wei Ning Ultrasound Information Management System (miis60pro, Hefei, China), four experienced physicians analyzed images for the presence or absence of plaques and classified them as stable or unstable. The images and assessments were reviewed by two chief physicians who resolved disagreements through discussion.

2.2 Deep learning models

2.2.1 Image preprocessing

We first manually cropped all images to a standard size in order to remove irrelevant background areas and retain only the ultrasound image region. We then normalized the data using the mean and standard deviation of all images in order to ensure that data followed a standard distribution.

To enhance the model’s ability to generalize and improve its robustness, we employed data augmentation techniques such as random horizontal flipping, random vertical flipping, and random rotation. In order to mitigate the overfitting issue, which could arise due to the presence of a small number of erroneous samples, we incorporated label smoothing into our strategy. This approach prevented the model from relying excessively on training samples, ensuring a more balanced learning process.

2.2.2 Development of the BCNN-ResNet algorithm

The BCNN-ResNet algorithm is designed to construct a model that includes autonomously cropping each image to leave only the carotid artery and potential plaques, as well as for the identification and classification of plaques. Before building this algorithm, we attempted to start with various existing CNN models and Transformer models, training them with our dataset. Due to the small size of the dataset, models with Transformer architecture performed poorly on this task. Among the common CNN models, the ResNet network was superior in feature extraction for ultrasound images, which are full of high-frequency noise, compared to other CNN models like VGG. Analyzing the feature heatmaps of the ResNet model, we found it had a significant advantage in recognizing the vascular system within ultrasound images (Lei et al., 2020; Cheng et al., 2022; Johri et al., 2022; Shokouhmand et al., 2023), where plaques are located.

Starting with ResNet-18 for this task, we found that ResNet-50, which has more parameters, improved the model’s performance, but the performance of ResNet-34 actually worsened. Therefore, we sought a method to increase the number of model parameters without increasing the model’s depth. Thus, we proposed the BCNN-ResNet network, using ResNet-50 as the backbone network, linearly combining features extracted by two ResNet-50 networks. Feature fusion is achieved by bilinearly combining two images of the same resolution as the original image, creating a mixed feature matrix that aligns pixels at corresponding positions in both images. The fused features are then input into a fully connected layer, which outputs the classification results. The first ResNet-50 network takes edge-extracted ultrasound images as input, retaining only the structural information of the image. The second ResNet-50 network uses the original ultrasound images as input to extract pixel information. The fusion of structural and pixel information enables finer image classification.

2.2.3 Modeling for detecting carotid plaques

For the classification network of plaque presence and absence (Figure 2), we built a BCNN network using two Res-Net50 networks but only a single input. Let the two networks perform feature extraction on the image separately, and then bilinear feature combination of the two features at the same location, stretch the mixed feature matrix into a vector, and perform moment normalization and L2 normalization on the vector to obtain the fused features, and finally use the fully connected layer for classification.

Figure 2

Figure 2. Schematic of the BCNN-ResNet algorithm. Flowchart of carotid plaque detection using a single-input algorithm.

2.2.4 Modeling for assessing plaque stability

For the stability classification task of plaques (Figure 3), we build a dual-input BCNN network on top of the single-input BCNN network to extract more obvious difference features to improve the model effect since the pixel difference between different plaques is smaller than that of the previous task. The dual-input network still uses two Res-Net50 networks as the backbone network, but has two inputs, which are ultrasonic image after edge extraction retaining only the structural information of the image; the second one uses the original ultrasound image as the input to extract the pixel information to complete the finer-grained image classification task. A detailed processing flowchart of the Model 2 network in Supplementary Figure S2.

Figure 3

Figure 3. Schematic of the BCNN-ResNet algorithm. Flowchart of plaque stability assessment using a dual-input algorithm.

2.2.5 Parameter setting

We trained the BCNN-ResNet algorithm using modeling for detecting carotid plaques training dataset and modeling for assessing plaque stability training dataset, the AdamW optimization algorithm and a batch size of 64, while dynamic learning rate adjustments were made through cosine annealing. Before we decided to use ResNet-34 as the backbone network, we tested whether to use pre-trained weights. The ResNet-34 network using completely random initialization parameters lags behind the AUC indicator by about 4% compared to the ResNet34 network pre-trained using imagenet. Therefore, the backbone network of two ResNet-34 models was initialized using pre-trained ImageNet parameters. We further refined the cross-entropy loss function to incorporate label smoothing, thereby enhancing model robustness. The algorithm was executed within the PyTorch machine learning framework. Training was performed on four Nvidia RTX 4000 GPUs.

2.3 Algorithm evaluation

2.3.1 Evaluation modeling for detecting carotid plaques and modeling for assessing plaque stability algorithms

Modeling for detecting carotid plaques and Modeling for assessing plaque stability algorithms were validated using an internal testing dataset from patients belonging to the same cohort as those used for training, as well as an external testing dataset from patients who were non-overlap with the training and internal testing dataset. In both cases, algorithm performance was assessed for the two tasks of plaque detection and plaque stability assessment in terms of the area under the AUC (the area under the ROC curve), accuracy, sensitivity, and specificity. Thresholds for defining whether a plaque was present or absent or whether it was stable or unstable were optimized using the Youden index. Where appropriate, results were reported with 95% confidence intervals (CIs).

2.3.2 Demonstration of modeling for assessing plaque stability algorithms using gradient-weighted class activation mapping

In order to describe the prediction from our Modeling for assessing plaque stability, we used Grad-CAM network visualization methods (Libby et al., 2019) to generate model’s heat-maps. Higher intensity areas in a heat-map correspond to regions in the input image that contribute more to the model’s prediction. These areas are where the model focused its attention during the classification process. Conversely, lower intensity areas are less influential. Our heat-maps are the same as most Grad-CAM heat-maps, using a color scale to represent the intensity of importance. Warmer colors like red and yellow indicate higher importance, while cooler colors like blue and green represent lower importance.

2.3.3 Statistical analyses

Continuous data showing a normal distribution were reported as mean ± standard deviation, and inter-group differences were assessed for significance using a Student’s t test. Continuous data showing a skewed distribution were reported as median (interquartile range), and inter-group differences were assessed using a non-parametric test such as the Mann–Whitney U test.

Categorical data were reported as n (%), and inter-group differences were assessed using a chi-squared test if n > 40 or Fisher’s exact test otherwise.

2.3.4 Ethics statement

The study was approved by the Ethics Review Board at Shanghai Eighth People’s Hospital (approval 2022-015-09-02) and the Xinhua Hospital affiliated with Dalian University (approval 2022-100-01). The requirement for informed consent was waived by the Ethics Review Boards.

3 Results

3.1 The study cohort

Modeling for detecting carotid plaques consisted 1761 ultrasound images from 1,165 participants, 510 with plaques, 655 without carotid plagues. Modeling for assessing plaque stability consisted 1,043 ultrasound images from 510 participants, 156 with stable plagues, 354 with unstable plagues.

Among the 1,165 participants included in the Modeling for detecting carotid plaques analysis, those with carotid plaques showed significantly higher Male, Age, Lipoprotein a, SBP, DBP, Uric acid, Apolipoprotein A1, Apolipoprotein B and significantly lower Apolipoprotein E, HDL-C, LDL-C, Total cholesterol than those without plaques (Table 1). Among the 510 participants included in the Modeling for assessing plaque stability final analysis, those with stable plaques showed significantly higher SBP, Apolipoprotein A1, Apolipoprotein E, HDL-C, Total cholesterol than those with unstable ones (Table 2).

Table 1

Table 1. Clinicodemographic comparison between patients with or without plaques in the modeling for detecting carotid plaques.

Table 2

Table 2. Clinical characteristics of patients for stable and unstable plaques in the modeling for assessing plaque stability.

3.2 Model performance in detection of carotid plaques

For detecting carotid plaques in the internal testing dataset, the BCNN-ResNet algorithm had an AUC of 0.989 (95% CI 0.998–0.840), accuracy of 95.97%, sensitivity 93.20% and specificity 99.21%. In the external testing dataset, the AUC of 0.951 (95% CI 0.962–0.939), with 86.38% accuracy, 95.30% sensitivity and 82.24% specificity (Figures 4A, 5A,B and Table 3).

Figure 4

Figure 4. Receiver operating characteristic curves to assess the ability of the BCNN-ResNet algorithm to (A) detect carotid plaques or (B,C) assess plaque stability in (B) single-input or (C) dual-input mode. The algorithm was assessed against the training dataset (red curves), internal testing dataset (blue) or external testing dataset (gold). The area under each curve is indicated in the legends at the bottom right of each panel.

Figure 5

Figure 5. Confusion matrices to assess the ability of the BCNN-ResNet algorithm to (A,B) detect carotid plaques in the (A) training dataset or (B) external testing dataset; (C,D) assess plaque stability in single-input mode in the (C) training dataset or (D) external testing dataset; or (E,F) assess plaque stability in dual-input mode in the (E) training dataset or (F) external testing dataset.

Table 3

Table 3. Performance of BCNN-ResNet algorithms in internal and external testing.

3.3 Model performance in assessment of carotid plaques stability

For assessing plaques as stable or unstable, the performance parameters of AUC 0.896 (95% CI 0.922–0.865), accuracy of 83.66%, sensitivity of 81.63% and specificity of 87.27% in internal testing dataset, and 0.854 (95% CI 0.889–0.830), 74.55, 68.52 and 89.49% in the external testing dataset (Figures 4B,C, 5C–F and Table 3). The combination of BCNN and Res-Net50, especially in dual-input mode, assessed plaque stability markedly better than other Res-Net architectures (Supplementary Table S1).

Original ultrasound images and gradient-weighted class activation maps from representative patients within Modeling for assessing plaque stability are shown in Figure 6.

Figure 6

Figure 6. Gradient-weighted class activation mapping to identify areas of ultrasound images that the BCNN-ResNet algorithm weighed more during its calculations. Original ultrasound images that served as input are shown in grayscale, while the neighboring image is colored according to where the algorithm “focused.”

4 Discussion

In this study, we built Modeling for detecting carotid plaques and Modeling for assessing plaque stability algorithms, we provide evidence that a novel combination of a BCNN and Res-Net architecture can accurately detect carotid plaques in ultrasound images, as well as assess their stability, which correlates with risk of stroke and other cardiovascular disorders (Selwaness et al., 2016; Saba et al., 2019, 2022; Van Der Toorn et al., 2022). Modeling for detecting carotid plaques and modeling for assessing plaque stability demonstrate excellent diagnostic performance. The proposal of this study provides a new and objective automatic quantitative approach for the detection of carotid plaque and the determination of plaque stability, which may provide an effective examination method for early screening of stroke in clinical practice.

For the ultrasound images of carotid, the ultrasound imaging effect has more noise compared with CT and other modalities, and the area of the plaque is smaller compared with the whole carotid image, the finer-grained image classification task, which is more suitable for the carotid ultrasound image classification task research. Our study justifies further work to optimize and develop the combined network into a tool for automated detection and analysis of carotid plaques, and it may help guide future efforts to integrate deep learning into complex image classification pathways in the clinic.

Our Modeling for detecting carotid plaques, the internal testing AUC was 0.989 (95% CI 0.998–0.840) and the external testing AUCwas 0.951 (95% CI 0.962–0.939), better than an AUC of 0.935 in previously reported deep learning method to identify plaque location (Li et al., 2021). A deep learning method has also been reported for assessing plaque area (Yu et al., 2021; Jain et al., 2022; Lin et al., 2022), which could help monitor plaque progression and regression. In our study, we identified the presence or absence of plaque in the carotid based on further identification of plaque stability, which is more clinically useful for clinical prediction of stroke than identifying the location and size of plaque, and unstable high-risk carotid plaque suggests a higher risk of stroke occurrence. Modeling for assessing plaque stability algorithms, the internal testing AUC (0.896), the external testing AUC (0.854), the internal testing AUC was better than previous existing models AUC (0.868) (We validated using the Modeling for Assessment Plaque stability dataset, see Supplementary Table S1). Our work, together with those previous studies, supports the idea that deep learning can accurately and reproducibly identify even small, subtle features in noisy ultrasound images. Indeed, deep learning appears to be superior to traditional machine learning algorithms for fine-grained interpretation of medical images (Biswas et al., 2021; Latha et al., 2022; van Dam-Nolen et al., 2022). In addition, machine learning requires more time and human involvement (van Veelen et al., 2022), for example, features need to be manually defined, which can make the algorithm’s classifications less reliable and robust to noise or patient heterogeneity. This observation provides a crucial understanding of the specific regions within the ultrasound imagery that significantly contribute to the predictions of the network. The aforementioned insight underscores the capacity of our model to discern and focus on clinically significant features such as carotid plaques, thereby making it a valuable tool for automated diagnosis and feature extraction. The results suggest that further integration of Grad-CAM into our deep learning model has the potential to enhance the interpretability and transparency of these predictive networks, an attribute of utmost importance in the application of deep learning in the medical imaging domain.

In our study, the AUC of the Modeling for detecting carotid plaques was 0.093 higher than that of the Modeling for assessing plaque stability, with a 11.75% higher sensitivity and a 11.94% higher specificity on the internal test. On the external test, the AUC of the Modeling for detecting carotid plaques was 0.093 higher than that of the Modeling for assessing plaque stability, with a 26.78% higher sensitivity and a 11.94% higher specificity. In classification tasks, the granularity of classification is a key factor that affects classification performance. The analysis suggests that in the two main tasks mentioned in this article, the task of plaque detection only requires a small number of features to achieve good results because the goal is simply to determine whether a plaque exists in the carotid artery. However, for the task of classifying the stability of carotid plaque, a more detailed classification of plaque types is required, and the model needs to learn deeper features for each type. This necessitates more parameters and stronger feature extraction capabilities in the model, which is a challenging task in data-limited scenarios. Consequently, the AUC and other indicators for this model are lower than for coarse-grained tasks. Therefore, we adopted dual-input BCNN network for identifying the stability of carotid plaques. To our knowledge, this model has advancements in recognizing carotid plaque ultrasound images and has not been reported previously.

Our study still has some limitations, the algorithm should be validated and further optimized in larger patient samples, such as through the integration of a transformer-based network. It may be useful to extend the algorithm to the assessment of lipid nuclei in unstable plaques, extent of calcification, or size of ulcerated plaque craters. These features have been linked to the risk of stroke. In future work, we will expand the external validation to include a wider range of locations and populations, which would make the findings more universally applicable. Another useful addition would be a study of how the model performs over time with the same patients, to assess its consistency and reliability.

5 Conclusion

In conclusion, we present a combined BCNN-ResNet algorithm that shows superior performance compared to other deep learning methods in detecting carotid plaques and assessing their stability. The BCNN network demonstrates advanced capabilities in recognizing carotid artery ultrasound images. Our BCNN-network outperforms previous models in determining the presence of carotid plaques and identifying their stability. The application of our algorithm could potentially streamline clinical workflows, facilitate clinical screening for carotid artery disease, and contribute to the prevention of stroke.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary material, further inquiries can be directed to the corresponding authors.

Ethics statement

The study was approved by the Ethics Review Board at Shanghai Eighth People’s Hospital (approval 2022-015-09-02) and the Xinhua Hospital affiliated with Dalian University (approval 2022-100-01). The studies were conducted in accordance with the local legislation and institutional requirements. Written informed consent for participation was not required from the participants or the participants’ legal guardians/next of kin because this is a retrospective study, we retrospectively analysed the collection of images and information needed for the modelling process. Written informed consent was not obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article because this is a retrospective study, we retrospectively analysed the collection of images and information needed for the modelling process.

Author contributions

LH: Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing. ZY: Conceptualization, Data curation, Investigation, Methodology, Software, Supervision, Visualization, Writing – original draft. YuW: Conceptualization, Data curation, Formal analysis, Software, Writing – review & editing. WC: Conceptualization, Data curation, Investigation, Methodology, Project administration, Software, Supervision, Visualization, Writing – review & editing. LD: Data curation, Formal analysis, Investigation, Methodology, Software, Validation, Writing – review & editing. YiW: Formal analysis, Investigation, Validation, Writing – review & editing. WY: Data curation, Formal analysis, Investigation, Writing – review & editing. XL: Data curation, Formal analysis, Investigation, Writing – review & editing. YZ: Conceptualization, Data curation, Project administration, Supervision, Validation, Writing – review & editing. YH: Project administration, Supervision, Validation, Writing – review & editing. ES: Funding acquisition, Project administration, Resources, Supervision, Writing – original draft, Writing – review & editing.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This work was provided financial support for the conduct of the research in study design by the Special Research Project of Smart Healthcare in Xuhui District, Shanghai (XHZH202108).

Conflict of interest

ZY, YuW, WC, and LD were employed by Infervision.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frai.2024.1321884/full#supplementary-material

References

Baradaran, H., and Gupta, A. (2021). Extracranial vascular disease: carotid stenosis and plaque imaging. Neuroimaging Clin. N. Am. 31, 157–166. doi: 10.1016/j.nic.2021.02.002

PubMed Abstract | Crossref Full Text | Google Scholar

Biswas, M., Saba, L., Omerzu, T., Johri, A. M., Khanna, N. N., Viskovic, K., et al. (2021). A review on joint carotid intima-media thickness and plaque area measurement in ultrasound for cardiovascular/stroke risk monitoring: artificial intelligence framework. J. Digit. Imaging 34, 581–604. doi: 10.1007/s10278-021-00461-2

PubMed Abstract | Crossref Full Text | Google Scholar

Bos, D., Arshi, B., van den Bouwhuijsen, Q. J. A., Ikram, M. K., Selwaness, M., Vernooij, M. W., et al. (2021). Atherosclerotic carotid plaque composition and incident stroke and coronary events. J. Am. Coll. Cardiol. 77, 1426–1435. doi: 10.1016/j.jacc.2021.01.038

Crossref Full Text | Google Scholar

Chaudhary, K., Poirion, O. B., Lu, L., and Garmire, L. X. (2018). Deep learning-based multi-omics integration robustly predicts survival in liver Cancer. Clin. Cancer Res. 24, 1248–1259. doi: 10.1158/1078-0432.CCR-17-0853

PubMed Abstract | Crossref Full Text | Google Scholar

Chen, Z. H., Lin, L., Wu, C. F., Li, C. F., Xu, R. H., and Sun, Y. (2021). Artificial intelligence for assisting cancer diagnosis and treatment in the era of precision medicine. Cancer Commun 41, 1100–1115. doi: 10.1002/cac2.12215

PubMed Abstract | Crossref Full Text | Google Scholar

Cheng, N., Ren, Y., Zhou, J., Zhang, Y., Wang, D., Zhang, X., et al. (2022). Deep learning-based classification of hepatocellular nodular lesions on whole-slide histopathologic images. Gastroenterology 162, 1948–1961. doi: 10.1053/j.gastro.2022.02.025

PubMed Abstract | Crossref Full Text | Google Scholar

Francois, D., Jean, M., Marie-France, G., Gilles, S., and Guy, C. (2011). Segmentation of plaques in sequences of ultrasonic B-mode images of carotid arteries based on motion estimation and a Bayesian model. IEEE Trans. Biomed. Eng. 58, 2202–2211. doi: 10.1109/TBME.2011.2127476

Crossref Full Text | Google Scholar

Hajhosseiny, R., Bahaei, T. S., Prieto, C., and Botnar, R. M. (2019). Molecular and nonmolecular magnetic resonance coronary and carotid imaging. Arterioscler. Thromb. Vasc. Biol. 39, 569–582. doi: 10.1161/ATVBAHA.118.311754

PubMed Abstract | Crossref Full Text | Google Scholar

Hu, S., Cui, B., Mlynash, M., Zhang, X., Mehta, K. M., and Lansberg, M. G. (2020). Stroke epidemiology and stroke policies in China from 1980 to 2017: a systematic review and meta-analysis. Int. J. Stroke 15, 18–28. doi: 10.1177/1747493019873562

PubMed Abstract | Crossref Full Text | Google Scholar

Jain, P. K., Sharma, N., Kalra, M. K., Johri, A., Saba, L., and Suri, J. S. (2022). Far wall plaque segmentation and area measurement in common and internal carotid artery ultrasound using U-series architectures: an unseen artificial intelligence paradigm for stroke risk assessment. Comput. Biol. Med. 149:106017. doi: 10.1016/j.compbiomed.2022.106017

PubMed Abstract | Crossref Full Text | Google Scholar

Johri, A. M., Nambi, V., Naqvi, T. Z., Feinstein, S. B., Kim, E. S. H., Park, M. M., et al. (2020). Recommendations for the assessment of carotid arterial plaque by ultrasound for the characterization of atherosclerosis and evaluation of cardiovascular risk: from the American Society of Echocardiography. J. Am. Soc. Echocardiogr. 33, 917–933. doi: 10.1016/j.echo.2020.04.021

Crossref Full Text | Google Scholar

Johri, A. M., Singh, K. V., Mantella, L. E., Saba, L., Sharma, A., Laird, J. R., et al. (2022). Deep learning artificial intelligence framework for multiclass coronary artery disease prediction using combination of conventional risk factors, carotid ultrasound, and intraplaque neovascularization. Comput. Biol. Med. 150:106018. doi: 10.1016/j.compbiomed.2022.106018

PubMed Abstract | Crossref Full Text | Google Scholar

Kerwin, W., Xu, D., Liu, F., Saam, T., Underhill, H., Takaya, N., et al. (2007). Magnetic resonance imaging of carotid atherosclerosis: plaque analysis. Top. Magn. Reson. Imaging 18, 371–378. doi: 10.1097/rmr.0b013e3181598d9d

PubMed Abstract | Crossref Full Text | Google Scholar

Kopczak, A., Schindler, A., Bayer-Karpinska, A., Koch, M. L., Sepp, D., Zeller, J., et al. (2020). Complicated carotid artery plaques as a cause of cryptogenic stroke. J. Am. Coll. Cardiol. 76, 2212–2222. doi: 10.1016/j.jacc.2020.09.532

Crossref Full Text | Google Scholar

Latha, S., Muthu, P., Dhanalakshmi, S., Kumar, R., Lai, K. W., and Wu, X. (2022). Emerging feature extraction techniques for machine learning-based classification of carotid artery ultrasound images. Comput. Intell. Neurosci. 2022:1847981. doi: 10.1155/2022/1847981

PubMed Abstract | Crossref Full Text | Google Scholar

LeCun, Y., Bengio, Y., and Hinton, G. (2015). Deep learning. Nature 521, 436–444. doi: 10.1038/nature14539

PubMed Abstract | Crossref Full Text | Google Scholar

Lei, Y., Tian, Y., Shan, H., Zhang, J., Wang, G., and Kalra, M. K. (2020). Shape and margin-aware lung nodule classification in low-dose CT images via soft activation mapping. Med. Image Anal. 60:101628. doi: 10.1016/j.media.2019.101628

PubMed Abstract | Crossref Full Text | Google Scholar

Li, L., Hu, Z., Huang, Y., Zhu, W., Wang, Y., Chen, M., et al. (2021). Automatic multi-plaque tracking and segmentation in ultrasonic videos. Med. Image Anal. 74:102201. doi: 10.1016/j.media.2021.102201

PubMed Abstract | Crossref Full Text | Google Scholar

Libby, P., Buring, J. E., Badimon, L., Hansson, G. K., Deanfield, J., Bittencourt, M. S., et al. (2019). Atherosclerosis. Nat. Rev. Dis. Primers 5:56. doi: 10.1038/s41572-019-0106-z

Crossref Full Text | Google Scholar

Lin, S. Y., Law, K. M., Yeh, Y. C., Wu, K. C., Lai, J. H., Lin, C. H., et al. (2022). Applying machine learning to carotid sonographic features for recurrent stroke in patients with acute stroke. Front Cardiovasc Med. 9:804410. doi: 10.3389/fcvm.2022.804410

PubMed Abstract | Crossref Full Text | Google Scholar

Molinari, F, Liboni, W, Pavanelli, E, Giustetto, P, Badalamenti, S, and Suri, JS. Accurate and automatic carotid plaque characterization in contrast enhanced 2-D ultrasound images. The annual international conference of the IEEE engineering in medicine and biology society. Lyon, France, (2007), 335–338.

Google Scholar

Murray, C. S. G., Nahar, T., Kalashyan, H., Becher, H., and Nanda, N. C. (2018). Ultrasound assessment of carotid arteries: current concepts, methodologies, diagnostic criteria, and technological advancements. Echocardiography 35, 2079–2091. doi: 10.1111/echo.14197

PubMed Abstract | Crossref Full Text | Google Scholar

Parish, S., Arnold, M., Clarke, R., Du, H., Wan, E., Kurmi, O., et al. (2019). Assessment of the role of carotid atherosclerosis in the association between major cardiovascular risk factors and ischemic stroke subtypes. JAMA Netw. Open 2:e194873. doi: 10.1001/jamanetworkopen.2019.4873

PubMed Abstract | Crossref Full Text | Google Scholar

Roy-Cardinal, M. H., Destrempes, F., Soulez, G., and Cloutier, G. (2019). Assessment of carotid artery plaque components with machine learning classification using Homodyned-K parametric maps and Elastograms. IEEE Trans. Ultrason. Ferroelectr. Freq. Control 66, 493–504. doi: 10.1109/TUFFC.2018.2851846

PubMed Abstract | Crossref Full Text | Google Scholar

Saba, L., Antignani, P. L., Gupta, A., Cau, R., Paraskevas, K. I., Poredos, P., et al. (2022). International Union of Angiology (IUA) consensus paper on imaging strategies in atherosclerotic carotid artery imaging: from basic strategies to advanced approaches. Atherosclerosis 354, 23–40. doi: 10.1016/j.atherosclerosis.2022.06.1014

PubMed Abstract | Crossref Full Text | Google Scholar

Saba, L., Saam, T., Jager, H. R., Yuan, C., Hatsukami, T. S., Saloner, D., et al. (2019). Imaging biomarkers of vulnerable carotid plaques for stroke risk prediction and their potential clinical implications. Lancet Neurol. 18, 559–572. doi: 10.1016/S1474-4422(19)30035-3

PubMed Abstract | Crossref Full Text | Google Scholar

Selwaness, M., Bos, D., van den Bouwhuijsen, Q., Portegies, M. L., Ikram, M. A., Hofman, A., et al. (2016). Carotid atherosclerotic plaque characteristics on magnetic resonance imaging relate with history of stroke and coronary heart disease. Stroke 47, 1542–1547. doi: 10.1161/STROKEAHA.116.012923

PubMed Abstract | Crossref Full Text | Google Scholar

Shokouhmand, A., Wen, H., Khan, S., Puma, J. A., Patel, A., Green, P., et al. (2023). Diagnosis of peripheral artery disease using backflow abnormalities in proximal recordings of accelerometer contact microphone (ACM). IEEE J. Biomed. Health Inform. 27, 274–285. doi: 10.1109/JBHI.2022.3218595

PubMed Abstract | Crossref Full Text | Google Scholar

Sun, R., Wang, L., Guan, C., Cao, W., and Tian, B. (2018). Carotid atherosclerotic plaque features in patients with acute ischemic stroke. World Neurosurg. 112, e223–e228. doi: 10.1016/j.wneu.2018.01.026

Crossref Full Text | Google Scholar

Tjoa, E., and Guan, C. (2021). A survey on explainable artificial intelligence (XAI): toward medical XAI. IEEE Trans Neural Netw Learn Syst. 32, 4793–4813. doi: 10.1109/TNNLS.2020.3027314

PubMed Abstract | Crossref Full Text | Google Scholar

van Dam-Nolen, D. H. K., Truijman, M. T. B., van der Kolk, A. G., Liem, M. I., Schreuder, F., Boersma, E., et al. (2022). Carotid plaque characteristics predict recurrent ischemic stroke and TIA: the PARISK (plaque at RISK) study. JACC Cardiovasc. Imaging 15, 1715–1726. doi: 10.1016/j.jcmg.2022.04.003

PubMed Abstract | Crossref Full Text | Google Scholar

Van Der Toorn, J. E., Bos, D., Ikram, M. K., Verwoert, G. C., van der Lugt, A., Ikram, M. A., et al. (2022). Carotid plaque composition and prediction of incident atherosclerotic cardiovascular disease. Circ` Cardiovasc. Imaging. 15:e013602. doi: 10.1161/CIRCIMAGING.121.013602

PubMed Abstract | Crossref Full Text | Google Scholar

van Veelen, A., van der Sangen, N. M. R., Henriques, J. P. S., and Claessen, B. (2022). Identification and treatment of the vulnerable coronary plaque. Rev. Cardiovasc. Med. 23:39. doi: 10.31083/j.rcm2301039

PubMed Abstract | Crossref Full Text | Google Scholar

Wang, X., and Qi, G. J. (2023). Contrastive learning with stronger augmentations. IEEE Trans. Pattern Anal. Mach. Intell. 45, 5549–5560. doi: 10.1109/TPAMI.2022.3203630

PubMed Abstract | Crossref Full Text | Google Scholar

Wang, L., Shao, A., Huang, F., Liu, Z., Wang, Y., Huang, X., et al. (2023). Deep learning-based semantic segmentation of non-melanocytic skin tumors in whole-slide histopathological images. Exp. Dermatol. 32, 831–839. doi: 10.1111/exd.14782

PubMed Abstract | Crossref Full Text | Google Scholar

Wang, Y. W., Wang, J. W., Yang, S. X., Qi, L. L., Lin, H. L., Zhou, Z., et al. (2021). Proposing a deep learning-based method for improving the diagnostic certainty of pulmonary nodules in CT scan of chest. Eur. Radiol. 31, 8160–8167. doi: 10.1007/s00330-021-07919-5

PubMed Abstract | Crossref Full Text | Google Scholar

Ying, C., Li, X., Lv, S., Du, P., Chen, Y., Fu, H., et al. (2022). T-SPOT with CT image analysis based on deep learning for early differential diagnosis of nontuberculous mycobacteria pulmonary disease and pulmonary tuberculosis. Int. J. Infect. Dis. 125, 42–50. doi: 10.1016/j.ijid.2022.09.031

PubMed Abstract | Crossref Full Text | Google Scholar

Yu, J., Zhou, Y., Yang, Q., Liu, X., Huang, L., Yu, P., et al. (2021). Machine learning models for screening carotid atherosclerosis in asymptomatic adults. Sci. Rep. 11:22236. doi: 10.1038/s41598-021-01456-3

Crossref Full Text | Google Scholar

Zhang, S., Yu, X., Gu, H., Kang, B., Guo, N., and Wang, X. (2022). Identification of high-risk carotid plaque by using carotid perivascular fat density on computed tomography angiography. Eur. J. Radiol. 150:110269. doi: 10.1016/j.ejrad.2022.110269

PubMed Abstract | Crossref Full Text | Google Scholar

Zhou, R., Guo, F., Azarpazhooh, M. R., Hashemi, S., Cheng, X., Spence, J. D., et al. (2021). Deep learning-based measurement of Total plaque area in B-mode ultrasound images. IEEE J. Biomed. Health Inform. 25, 2967–2977. doi: 10.1109/JBHI.2021.3060163

PubMed Abstract | Crossref Full Text | Google Scholar