Histopathology-Based Deep-Learning Predicts Atherosclerotic Lesions in Intravascular Imaging

Background: Optical coherence tomography is a powerful modality to assess atherosclerotic lesions, but detecting lesions in high-resolution OCT is challenging and requires expert knowledge. Deep-learning algorithms can be used to automatically identify atherosclerotic lesions, facilitating identification of patients at risk. We trained a deep-learning algorithm (DeepAD) with co-registered, annotated histopathology to predict atherosclerotic lesions in optical coherence tomography (OCT).

Methods: Two datasets were used for training DeepAD: (i) a histopathology data set from 7 autopsy cases with 62 OCT frames and co-registered histopathology for high quality manual annotation and (ii) a clinical data set from 51 patients with 222 OCT frames in which manual annotations were based on clinical expertise only. A U-net based deep convolutional neural network (CNN) ensemble was employed as an atherosclerotic lesion prediction algorithm. Results were analyzed using intersection over union (IOU) for segmentation.

Results: DeepAD showed good performance regarding the prediction of atherosclerotic lesions, with a median IOU of 0.68 ± 0.18 for segmentation of atherosclerotic lesions. Detection of calcified lesions yielded an IOU = 0.34. When training the algorithm without histopathology-based annotations, a performance drop of >0.25 IOU was observed. The practical application of DeepAD was evaluated retrospectively in a clinical cohort (n = 11 cases), showing high sensitivity as well as specificity and similar performance when compared to manual expert analysis.

Conclusion: Automated detection of atherosclerotic lesions in OCT is improved using a histopathology-based deep-learning algorithm, allowing accurate detection in the clinical setting. An automated decision-support tool based on DeepAD could help in risk prediction and guide interventional treatment decisions.

Introduction

Coronary artery disease is the leading cause of death worldwide, accounting for the majority of acute coronary syndromes and sudden cardiac deaths. Identification of atherosclerotic tissue might allow stratification of patients at risk for future coronary events (1–3). Invasive assessment of the coronary arteries using high-resolution intravascular imaging has emerged as an important tool for identifying atherosclerotic lesions (4), as the close proximity of the imaging catheter allows a more precise and high-resolution visualization of the vascular tissue compared to non-invasive modalities (5). The main advantage of optical coherence tomography (OCT) in comparison to other imaging modalities such as intravascular ultrasound (IVUS) is its superb resolution of 10–20 μm with a tissue penetration of approximately 2–3 mm, which enables clear visualization of atherosclerotic components such as lipid tissue with foam cell infiltration or calcifications (5, 6). However, assessment and evaluation of OCT is currently based on clinical expertise from skilled interventionalists and pitfalls in plaque characterization can lead to misclassification (7, 8). Additionally, each volume of OCT images obtained in one measurement, also called a pullback, generates a vast amount of data, precluding from analog assessment of single frames. Due to the high clinical relevance and the cumbersome manual review, automated or semi-automated approaches for preliminary evaluation of OCT images is of great interest for the interventional community (9–11). For intravascular imaging, several groups demonstrated the possibility of automated plaque characterization using semi-automated approaches such as quantification of various optical signal properties (e.g. light attenuation) (12–15). The feasibility of using deep-learning algorithms for detection and visualization of atherosclerotic plaque components was also demonstrated recently (9, 11, 16–19). In this work, we developed DeepAD, a deep-learning algorithm trained on data with histopathology-based annotations from autopsy specimens as well as clinical OCTs for prediction of atherosclerotic lesions, and evaluated it in dedicated real-world cases (Figure 1).

FIGURE 1

Figure 1. DeepAD for prediction of atherosclerotic lesions from OCT. (A) Two data sets were used for training DeepAD: 222 OCT frames from 51 patients with manual annotations based on clinical expertise, and 62 OCT frames from 7 patients with annotations based on co-registered histopathology. (B) DeepAD learned to accurately predict atherosclerotic lesions from OCT images and was evaluated on patients with histopathology-based annotations.

Methods

Histopathology and ex vivo OCT Imaging

Autopsy samples of coronary arteries were acquired from the department of pathology at Klinikum Rechts der Isar in accordance with federal state law and after approval from relatives (ethical approval number: 291/18 S). Coronary arteries were then cut by an experienced pathologist according to the standard AHA classification. OCT imaging of the single segments was then performed as previously described (15): After careful preparation, vessels were wired with a 0.014-inch guidewire over which the OCT catheter (2.7-F, St. Jude Medical, St. Paul, Minnesota) was advanced. An imaging pullback was performed from the distal to the proximal arterial segments while simultaneously flushing the vessel with contrast to improve imaging quality (pullback speed 5 mm/s = 120 frames/s). Afterwards, arteries were cut at 3 mm intervals and stained with haematoxylin-eosin (H.E.) and Movat-Pentachrome. The presence of different atherosclerotic plaque types and plaque components was evaluated by an expert in cardiovascular pathology (MJ) according to the classification by Virmani et al. (20), (see Table 1). Co-registration of OCT frames and histological sections of human autopsy samples was achieved as previously described (21) by first, considering only the proximal, middle or distal part of the OCT pullback for the respective slide and second, visual alignment of anatomical landmarks (calcifications, side branches, lumen contour) within each part of the segment.

TABLE 1

Table 1. Histopathological data set (n = 62 histopathology images).

Clinical Data Set

OCT pullbacks from the OCT database at the German Heart Center Munich (including all patients undergoing intravascular imaging with clinical indication for OCT during coronary angiography) were screened. OCT imaging was performed according to current recommendations (22) with commercially available OCT systems (Dragonfly DF-OCT-catheter combined with the C7-XRTM imaging system; LightLab Imaging Inc., Westford MA, USA). Pullbacks were assessed for the presence of atherosclerotic plaque features as previously described (23) and included fibrous, lipid and calcified plaques (see Table 2). A total of 222 frames from 51 patients were used for analysis. In case of stented vessels, proximal and distal regions outside the stent were used. For baseline characteristics, (see Table 3).

TABLE 2

Table 2. Clinical data set (n = 222 OCT frames).

TABLE 3

Table 3. Baseline characteristics of clinical data set.

Manual Annotation of OCT Frames

OCT images were transferred to an offline working station and transformed into 8bit RGB images. Labeling of OCT frames was achieved using the freeware tool LabelMe (24). In each frame, the suspected plaque area was identified. Plaque area was measured based on histopathology and by tracing the leading edge in OCT images using a polygonal line tool, respectively. Plaque area was defined as the presence of lipid accumulation, necrotic core with or without foam cell infiltration and/or calcification (Figure 2). To avoid pitfalls as reported in (7), we included the guidewire artifact into the label “background”.

FIGURE 2

Figure 2. Lesion segmentation vs. a-line classification. (A) Lesion segmentation: Atherosclerotic lesions are annotated in OCT frames with or without histopathological information (blue = lumen, gray= vessel wall). Performance of the segmentation approach is evaluated via IOU, measuring the overlap between the labeled “ground truth” and the “prediction” of the algorithm. IOU ranges between 0 and 1 (B) a-line classification. Based on the manual annotation, 360 a-line predictions are evaluated per OCT frame regarding the presence or absence of atherosclerotic lesions and can be summarized in a confusion matrix.

DeepAD

DeepAD is an ensemble of semantic segmentation networks performing pixelwise classification with respect to atherosclerotic lesions. In addition to histopathology-based OCT annotations, we employed so-called weakly supervised training by using OCT images annotated without histopathology. While histopathology-based annotations represent the gold standard, annotations based on OCT images contain valuable information for learning to segment atherosclerotic lesions, however, remain limited in resolution and diagnosis, hence providing a weak supervisory signal to the training (25). We trained and selected several models in so-called ensembles, an effective method to avoid overfitting on small data sets (26). All the models in the ensemble were created with the same network architecture. The differences in the models were the weight initializations of the networks, training batch sampling as well as optimization parameters. As no random seeds were set when training the models, these parameters were assigned different values at the start of each training round. This is a commonly used strategy to achieve different models as the network training of Deep Learning algorithms is stochastic and typically converges to different local minima (27). At inference time, all model predictions were averaged to obtain the final prediction for a given pixel. For further details on network architecture, hyperparameters, exact ensembling, (see Supplementary Material).

Evaluation on Unseen Patients

To leverage all the available data, DeepAD was trained several times using “leave one patient out” cross-validation. Specifically, for each training round, all histopathology-annotated OCT images from one patient were held out for testing. The remaining data was divided into a 80 % training and 20 % validation split, with no patient overlap between groups. The model was then trained with the training set and tested on the validation set. For each model in the ensemble, the model weights that performed best on the validation set were then tested on the held-out patient. This process was then repeated until each histopathology-annotated patient had been held out for testing once. The OCT images annotated without histopathology were never included in the test set.

Segmentation

Segmentation describes the task of linking certain regions or pixels within an image to a specific class label (here: lumen, lesion, other). We evaluated the segmentation performance using the intersection over union (IOU, also called Jaccard index), a common metric for evaluation of the quality of segmentation algorithms (28). The IOU is calculated as the overlap between the manual annotation (i.e. the “ground truth”) with the prediction from the algorithm divided by the union (see Figure 2A):

$\begin{array}{c}\text{IOU}=\text{Area of overlap}/\text{Area of union}.\end{array}$

IOU can range from 0 (no overlap between ground truth and prediction) to 1 (complete overlap between ground truth and prediction; Figure 2A).

a-Line Classification

In contrast to segmentation (i.e. the pixel-wise classification), a-line classification computes a binary output regarding the presence of a certain manually annotated class or label in an a-line (e.g. a lesion). In our work, one a-line was defined as one angular direction originating from the center of the lumen. For each OCT frame, 360 a-lines represented the whole 360 ° vessel circumference. Classification was evaluated on a binary level, with a-line being positive if >3 pixels along the a-line contained atherosclerotic lesion and negative if no pixels along the a-line contained atherosclerotic tissue, as defined in previous works (11). Positive a-line signal (indicating presence of atherosclerotic lesion) was visualized by a green line (arch) in the respective vessel circumference. For results of a-line classification, please see Supplementary Material and Supplementary Figure 1.

Results

Histopathology and Clinical Data set Characteristics

In the histopathology data set, a total of 62 lesions from 7 patients were identified according to the classification by Virmani et al. (20) and manually co-registered with OCT (see Table 1 for details regarding the different plaque types) as previously described (21). For the clinical data set, a total of 222 OCT frames were identified as suitable for analysis (see Table 2 for details on the different plaque types). For examples from both datasets (see Figure 3). The histopathology-based annotations as well as the clinical annotations were annotated independently by two clinical and histopathology experts. The estimated interobserver variability for manual annotation of OCT frames with and without histopathology-based annotations was measured as the IOU between two independent expert observers. The median interobserver variability was 0.58 without and 0.71 with histopathology. Thus, the agreement was considerably higher when histopathology-based annotations were available. We also measured the alignment between annotations on the same scans with and without histopathology, being 0.61 across the two annotators. This showed that a difference in results was observed when annotating with or without histopathology.

FIGURE 3

Figure 3. Examples from the histopathology data set and the clinical data set. (A) Histopathology of two atherosclerotic lesions (fibroatheroma with necrotic core covered by fibrous cap) with manual annotation (marked by red dashed line) and co-registered OCT frames. (B) Three examples of atherosclerotic lesions in clinical OCT images (marked by red dashed line), based on clinical judgement without underlying histopathology. Left: fibrotic plaque, middle: lipid plaque, right: calcified plaque.

DeepAD Accurately Predicts Atherosclerotic Tissue in Unseen Patients

DeepAD predicted atherosclerotic lesions with a median IOU of 0.68 ± 0.18 across all test patients (Figure 4A). Good and moderate performing examples (Figure 4B) illustrate that the DeepADs is highly accurate in the localization of atherosclerotic lesions across the lumen circumference as well as the difficulty of correctly estimating the extent of this lesion beyond the lumen border (see e.g. Figure 4B, moderate example). Figure 4B (lower row) illustrates a low performing prediction from DeepAD. Here DeepAD falsely predicts a lesion below the lumen which is not supported by the manual histopathology-based annotation. Overall, DeepAD predicts lesions with an IOU of 0.03 lower than the median IOU agreement between two observers, showing close to on par segmentations with expert annotators. To evaluate the importance of using histopathology-based OCT annotations, all OCT images in the histopathology data set were additionally annotated without histopathology by two independent observers. The algorithm was then trained with a dataset of the exact same size, but without histopathology-based annotations. The influence and importance of using histopathology-based annotation in the creation of the DeepAD algorithm was measured as the discrepancy of these models' performance with respect to the histopathology-based annotations. When the algorithm was trained using only annotations without histopathology, the median performance on the test set dropped by more than 0.25 IOU from 0.68 ± 0.18 to 0.42 ± 0.18 when predicting atherosclerotic lesion tissue (Figure 4A). This corresponds to a performance drop of around 33 %. In Figure 4B the predictions from DeepADs good, moderate and low performing cases are also shown for the algorithm trained without histopathology. In this case the segmentation algorithm yields false positive predictions (Figure 4B good performing example) and less extensive segmentation beyond the lumen border (Figure 4B median example). In the low performing example in Figure 4B both algorithms output similar false positive tissue prediction.

FIGURE 4

Figure 4. DeepAD predicts atherosclerotic lesion tissue on unseen test patients with good performance. (A) Intersection over union (IOU) scores across test sets for atherosclerotic tissue predictions from the OCT only and histopathology-based algorithm (DeepAD), here visualized for every test sample as a dot in the violin plot. (B) Examples of predicted atherosclerotic lesions from OCT and histopathology based algorithm: good performance prediction (top, 0.75 IOU, almost all of the region annotated in “ground truth” is predicted by DeepAD), moderate performance (middle, 0.66 IOU, most of the region annotated in “ground truth” is predicted by DeepAD) and low performance (bottom, 0.29 IOU, false-positive prediction of a region from 6-9 o'clock which was not annotated in “ground truth”).

Prediction of Calcification by DeepAD

Prediction of calcifications in our data set yielded an average IOU = 0.34. Figure 5 shows representative examples of good, intermediate and low performance. In most cases calcification was correctly localized by DeepAD but incompletely predicted (i.e. not all of the calcification was predicted by DeepAD).

FIGURE 5

Figure 5. Prediction of calcification by DeepAD. Examples for “good” (upper row), “intermediate” (middle row), and “low” (lower row) prediction of calcification using DeepAD (dark blue = lesion, green=calcification, light blue = lumen). Note that with an average IOU = 0.34, most calcification is localized correctly, however with incomplete detection.

Application of DeepAD in Clinical Cohort

To illustrate the use of the DeepAD in real-world clinical practice, atherosclerotic lesion prediction was demonstrated in 11 cases of patients who presented to the German Heart Center and underwent coronary angiography and intravascular imaging with OCT. Performance of the algorithm was retrospectively compared against manual analysis of each pullback by an expert regarding the presence or absence of atherosclerotic lesions as well as correct spatial prediction of the DeepAD on a frame-level. 3D-rendering of the complete pullback allowed quick and intuitive visualization of regions with atherosclerotic lesions in red, (see Figure 6A). A total of 3284 frames were analyzed and high agreement was seen between automated prediction by DeepAD and manual analysis (mean agreement 88%, 2898 of 3284 frames), with good sensitivity, specificity and accuracy (86.8, 82.9 and 85.8%, see Table 4). DeepAD tended to slightly underestimate the presence of atherosclerotic lesions (66 vs. 71% by manual analysis). Representative frames of false-positive predictions are shown in Supplementary Figure 2. For examples on detection of calcification, (see Figure 6B).

FIGURE 6

Figure 6. (A) Application of DeepAD for lesion detection in clinical cases. 3D reconstruction of OCT pullback visualizes algorithm-based lesions detection (red). Representative cross-sections of healthy (green) and diseased (red lines) areas are shown with respective lesion detection by the DeepAD. (B) Subdifferentiation of calcified lesions by DeepAD. Representative OCT frames and corresponding predictions by DeepAD of 3 clinical cases from the clinical cohort (see Table 4). Case 9 mostly has fibrolipidic lesions without any calcifications while case 10 shows heavily calcified lesions (note that DeepAD is able to detect and differentiate circumferential and spotty calcifications). Case 11 shows almost no atherosclerotic lesions.

TABLE 4

Table 4. Application of DeepAD in clinical cohort (n=11 cases) and comparison with manual analysis.

Discussion

The goal of our study was the development of a fully-automated, deep-learning algorithm trained on histopathology and clinical data for the prediction of atherosclerotic lesions in intravascular OCT. The main findings of our work are:

I DeepAD is able to predict atherosclerotic lesions in intravascular OCT images when trained on data from histopathology and clinical imaging, with an IOU = 0.68 (±0.18).

II Although prediction of calcification was lower (IOU = 0.34), calcified lesions were localized correctly in most cases (circumferential and spotty).

III Using histopathology-based annotations helped in training the algorithm over using only clinical annotations.

IV Clinical applicability of DeepAD was demonstrated in a small clinical cohort of 11 cases.

Application of deep-learning technology has the ability to transform the biomedical field in the 21th century. Due to significant advancements in access and storage of big data, and the availability of adequate processing power, use of algorithms embedded in artificial intelligence is likely to have a considerable impact on clinical practice toward more standardized and personalized patient care (29). The need for promoting artificial intelligence in the field of intravascular imaging is supported by the fact that the consequenes of advanced and undetected coronary artery disease are still detrimental, with cardiovascular disease being the leading cause of death worldwide and acute coronary syndromes or stroke accounting for 85% of these deaths (30). This hypothesis is supported by two large-scale studies: using virtual-histopathology IVUS (VH-IVUS) in 697 patients presenting with ACS, Stone et al. showed that lesions characterized as thin-cap fibroatheroma, although being angiographically mild, were responsible for the majority of non-culprit coronary events after 3 years (4). Similar, Prati et al. found that the presence of four high-risk plaque features identified by OCT (minimal lumen area <3.5 mm₂ fibrous cap thickness <75 mm, lipid arc circumferential ex- tension >180° and infiltration with macrophages) was associated with a higher risk of coronary events in 1,003 patients (31). Although all clinically used intracoronary imaging modalities (IVUS, OCT and NIRS) possess inherent limitations, the availability and application of high-resolution intracoronary imaging will likely increase with time (8). Most recently, NIRS (using near-infrared spectroscopy to visualize lipid-rich plaques) has been shown to detect patients at a higher risk for subsequent coronary events (32), being the first intracoronary imaging modality with the ability to provide predictive assessment of atherosclerotic coronary lesions. In our work, DeepAD predicted atherosclerotic lesions in general with high accuracy. While the identification of vulnerable plaques might be desirable and could be regarded as the “holy grail” in cardiovascular imaging, it may be argued that the concept of a single “vulnerable” plaque leading to coronary events might be overly simplistic and that rather the patient himself should be considered vulnerable. In this regard, detection of atherosclerosis burden might help in stratification of patients as the presence of atherosclerosis correlates with cardiovascular events (33, 34). For example, prospective studies have shown that identification of coronary artery calcification can be used for risk prediction in patients with suspected coronary artery disease and is therefore utilized in clinical risk scores (35). This seems contradictory at first as it is known from studies on vascular biology that calcification per se is an important mechanism of plaque stabilization: while macrophages are undergoing apoptosis, microcalcifications are occurring as a consequence of cell death (36). Eventually, the whole necrotic core might calcify over time, leading to stabilization and passivation of high-risk plaque regions (36). Nevertheless, calcification is closely correlating with overall atherosclerotic burden and therefore predicts mortality in patients with suspected coronary artery disease (37). Hence, similarities to our approach of using deep learning for detection of atherosclerotic burden with OCT are obvious: DeepAD enables identification of coronary atherosclerosis, which can be visualized using OCT but often pose a challenge in daily clinical practice for inexperienced clinician.

When detecting calcification, performance of DeepAD yielded an IOU = 0.34 (±0.07). While this is definitely inferior to the detection of atherosclerotic lesions in general (IOU = 0.68), we demonstrate that in most cases, calcification is correctly localized by DeepAD but incompletely predicted (i.e. not the complete extent of the calcification is predicted by DeepAD). For the interventionalist, this limitation might be neglectable as the sheer presence of calcification (even though not completely detected) is highly informative. This in turn might influence interventional treatment strategy (use of high-pressure or cutting-balloon, pre- or post-dilatation etc.) to deliver an optimal procedural result and avoid e.g. incomplete stent expansion.

With respect to previous work, only one study used co-registered histopathology images with OCT images for plaque segmentation (16). While our work is relying on a deep-learning approach to segment atherosclerotic lesions, the work by He at al. used ex vivo carotid plaque tissue samples and different, carefully handcrafted features with a decision tree. In our approach, CNN learns all filters automatically without any human supervision, hence a decision tree is not needed this way. Besides, deep-learning methods that do not use histopathology images have a limitation on the annotation quality: When segmenting without the help of histology images, only features which are clearly visible in the OCT image will be annotated and tissue that reaches further out will probably be missed. With the help of histopathology, these regions can be annotated more accurately. This ideally enables DeepAD to predict even hard-to-detect features that likely would have been missed if trained on clinical OCTs only.

Ultimately, DeepAD should be regarded as a decision-supporting tool which can aid in the clinical setting by providing a more standardized characterization of OCT with the potential to quickly visualize atherosclerotic burden in the cath lab. Whether this may be used for risk prediction has to be validated in dedicated future studies. In conclusion, our work highlights the value of using artificial intelligence in the field of intracoronary imaging. By providing an easily applicable tool based on a deep-learning algorithm trained with histopathology-annotated data as well as with clinical expertise, DeepAD can ultimately improve the diagnostic performance of intracoronary optical coherence tomography in detecting atherosclerotic lesions.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, upon reasonable request.

Ethics Statement

The studies involving human participants were reviewed and approved by Ethical Committee of TU München, Ethical Approval Number: 291/18 S. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.

Funding

This work was supported through a scientific grant by the German Cardiac Society (Grant Number: 16/2020). CM has received funding from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation programme (Grant agreement No. 866411).

Conflict of Interest

MJ reports personal fees from Biotronik, personal fees from Orbus Neich, grants and personal fees from Boston Scientific, grants and personal fees from Edwards, personal fees from Recor, personal fees from Astra Zeneca, grants from Amgen, personal fees from Abbott, personal fees from Shockwave, grants from Infraredx, grants from Cardiac Dimensions outside the submitted work.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fcvm.2021.779807/full#supplementary-material

Abbreviations

CAD, Coronary artery disease; CNN, Convolutional neural network; CVRF, Cardiovascular risk factors; DES, drug-eluting stent; H.E., Hematoxylin and eosin; IDDM, insulin-dependent diabetes mellitus; IOU, intersection over union; IVUS, intravascular ultrasound; LAD, Left anterior descending artery; LCA, Left coronary artery; LCX, Left circumflex artery; M.P., Movat-pentachrome; NIRS, near-infrared spectroscopy; OCT, Optical coherence tomography; PCI, Percutaneous coronary intervention; RCA, Right coronary artery.

References

1. Neumann F-J, Sousa-Uva M, Ahlsson A, Alfonso F, Banning AP, Benedetto U, et al. 2018 ESC/EACTS Guidelines on myocardial revascularization. The Task Force on myocardial revascularization of the European Society of Cardiology (ESC) and European Association for Cardio-Thoracic Surgery (EACTS). G Ital Cardiol. (2019) 20:1S−61S. doi: 10.1093/eurheartj/ehy394

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Thygesen K, Alpert JS, Jaffe AS, Chaitman BR, Bax JJ, Morrow DA, et al. Fourth universal definition of myocardial infarction (2018). Kardiol Pol. (2018) 76:1383–415. doi: 10.1161/CIR.0000000000000617

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Benjamin EJ, Virani SS, Callaway CW, Chamberlain AM, Chang AR, Cheng S, et al. Heart disease and stroke statistics−2018 update: a report from the american heart association. Circulation. (2018) 137:e67–492. doi: 10.1161/CIR.0000000000000573

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Stone GW, Maehara A, Lansky AJ, de Bruyne B, Cristea E, Mintz GS, et al. A prospective natural-history study of coronary atherosclerosis. New Engl J Med. (2011) 364:226–35. doi: 10.1056/NEJMoa1002358

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Bezerra HG, Costa MA, Guagliumi G, Rollins AM, Simon DI. Intracoronary optical coherence tomography: a comprehensive review clinical and research applications. JACC Cardiovasc Interv. (2009) 2:1035–46. doi: 10.1016/j.jcin.2009.06.019

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Jang I-K, Bouma BE, Kang D-H, Park S-J, Park S-W, Seung K-B, et al. Visualization of coronary atherosclerotic plaques in patients using optical coherence tomography: comparison with intravascular ultrasound. J Am Coll Cardiol. (2002) 39:604–9. doi: 10.1016/S0735-1097(01)01799-5

PubMed Abstract | CrossRef Full Text | Google Scholar

7. van Soest G, Regar E, Goderie TPM, Gonzalo N, Koljenović S, van Leenders GJLH, et al. Pitfalls in plaque characterization by OCT: image artifacts in native coronary arteries. JACC Cardiovasc Imaging. (2011) 4:810–3. doi: 10.1016/j.jcmg.2011.01.022

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Koskinas KC, Nakamura M, Räber L, Colleran R. Current use of intracoronary imaging in interventional practice—results of a European association of percutaneous cardiovascular interventions (EAPCI) and Japanese Association of Cardiovascular Interventions and Therapeutics (CVIT) Clinical Practice Survey. Circulation. (2018) 82:1360–8. doi: 10.1253/circj.CJ-17-1144

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Gessert N, Lutz M, Heyder M, Latus S, Leistner DM, Abdelwahed YS, et al. Automatic plaque detection in IVOCT pullbacks using convolutional neural networks. IEEE Trans Med Imaging. (2019) 38:426–34. doi: 10.1109/TMI.2018.2865659

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Gharaibeh Y, Dong P, Prabhu D, Kolluru C, Lee J, Zimin V, et al. Deep learning segmentation of coronary calcified plaque from intravascular optical coherence tomography (IVOCT) images with application to finite element modeling of stent deployment. In: Medical Imaging 2019: Image-Guided Procedures, Robotic Interventions, and Modeling. International Society for Optics and Photonics. (2019). p. 109511C. doi: 10.1117/12.2515256

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Lee J, Prabhu D, Kolluru C, Gharaibeh Y, Zimin VN, Bezerra HG, et al. Automated plaque characterization using deep learning on coronary intravascular optical coherence tomographic images. Biomed Opt Express. (2019) 10:6497–515. doi: 10.1364/BOE.10.006497

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Phipps JE, Vela D, Hoyt T, Halaney DL, Jacob Mancuso J, Maximilian Buja L, et al. Macrophages and intravascular OCT bright spots: a quantitative study. JACC Cardiovasc Imaging. (2015) 8:63–72. doi: 10.1016/j.jcmg.2014.07.027

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Vito LD, Di Vito L, Agozzino M, Marco V, Ricciardi A, Concardi M, et al. Identification and quantification of macrophage presence in coronary atherosclerotic plaques by optical coherence tomography. Eur Heart J Cardiovasc Imaging. (2015) 16:807–13. doi: 10.1093/ehjci/jeu307

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Gnanadesigan M, Kameyama T, Karanasos A, van Ditzhuijzen N, van der Sijde J, van Geuns R-J, et al. Automated characterisation of lipid core plaques in vivo by quantitative optical coherence tomography tissue type imaging. EuroIntervention. (2016) 12:1490–7. doi: 10.4244/EIJ-D-15-00320

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Nicol P, Hoppman P, Euller K, Xhepa E, Lenz T, Rai H, et al. Validation and application of OCT tissue attenuation index for the detection of neointimal foam cells. Int J Cardiovasc Imaging. (2021) 37:25–35. doi: 10.1007/s10554-020-01956-9

PubMed Abstract | CrossRef Full Text | Google Scholar

16. He C, Li Z, Wang J, Huang Y, Yin Y, Li Z. Atherosclerotic plaque tissue characterization: an OCT-based machine learning algorithm with ex vivo validation. Front Bioeng Biotechnol. (2020) 8:749. doi: 10.3389/fbioe.2020.00749

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Prabhu D, Bezerra H, Kolluru C, Gharaibeh Y, Mehanna E, Wu H, et al. Automated a-line coronary plaque classification of intravascular optical coherence tomography images using handcrafted features and large datasets. J Biomed Opt. (2019) 24:1–15. doi: 10.1117/1.JBO.24.10.106002

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Yong YL, Tan LK, McLaughlin RA, Chee KH, Liew YM. Linear-regression convolutional neural network for fully automated coronary lumen segmentation in intravascular optical coherence tomography. J Biomed Opt. (2017) 22:1–9. doi: 10.1117/1.JBO.22.12.126005

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Lee J, Prabhu D, Kolluru C, Gharaibeh Y, Zimin VN, Dallan LAP, et al. Fully automated plaque characterization in intravascular OCT images using hybrid convolutional and lumen morphology features. Sci Rep. (2020) 10:1–3. doi: 10.1038/s41598-020-59315-6

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Virmani R, Kolodgie FD, Burke AP, Farb A, Schwartz SM. Lessons from sudden coronary death: a comprehensive morphological classification scheme for atherosclerotic lesions. Arterioscler Thromb Vasc Biol. (2000) 20:1262–75. doi: 10.1161/01.ATV.20.5.1262

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Lutter C, Mori H, Yahagi K, Ladich E, Joner M, Kutys R, et al. Histopathological differential diagnosis of optical coherence tomographic image interpretation after stenting. JACC Cardiovasc Interv. (2016) 9:2511–23. doi: 10.1016/j.jcin.2016.09.016

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Tearney GJ, Regar E, Akasaka T, Adriaenssens T, Barlis P, Bezerra HG, et al. Consensus standards for acquisition, measurement, and reporting of intravascular optical coherence tomography studies. J Am Coll Cardiol. (2012) 59:1058–72.

PubMed Abstract | Google Scholar

23. Ueki Y, Räber L, Otsuka T, Rai H, Losdat S, Windecker S, et al. Mechanism of drug-eluting absorbable metal scaffold restenosis: a serial optical coherence tomography study. Circ Cardiovasc Interv. (2020) 13:e008657. doi: 10.1161/CIRCINTERVENTIONS.119.008657

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Wada K,. labelme: image polygonal annotation with python. (2016). Available online at: http://labelme.csail.mit.edu/guidelines.html

Google Scholar

25. Zhou Z-H A. brief introduction to weakly supervised learning. Natl Sci Rev. (2017) 5:44–53. doi: 10.1093/nsr/nwx106

CrossRef Full Text | Google Scholar

26. Zahavy T, Kang B, Sivak A, Feng J, Xu H, Mannor S. Ensemble robustness and generalization of stochastic deep learning algorithms. arXiv [cs.LG]. (2016). Available online at: http://arxiv.org/abs/1602.02389

Google Scholar

27. Ganaie MA, Hu M, Tanveer M, Suganthan PN. Ensemble deep learning: A review. ArXiv. (2021). abs/2104.02395.

Google Scholar

28. Minaee S, Boykov YY, Porikli F, Plaza AJ, Kehtarnavaz N, Terzopoulos D. Image segmentation using deep learning: a survey. IEEE Trans Pattern Anal Mach Intell. (2021). Available online at: http://dx.doi.org/10.1109/TPAMI.2021.3059968

PubMed Abstract | Google Scholar

29. Schmidt-Erfurth U, Sadeghipour A, Gerendas BS, Waldstein SM, Bogunović H. Artificial intelligence in retina. Prog Retin Eye Res. (2018) 67:1–29. doi: 10.1016/j.preteyeres.2018.07.004

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Timmis A, Townsend N, Gale CP, Torbica A, Lettino M, Petersen SE, et al. European society of cardiology: cardiovascular disease statistics 2019. Eur Heart J. (2020) 41:12–85. doi: 10.1093/eurheartj/ehz859

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Prati F, Romagnoli E, Gatto L, La Manna A, Burzotta F, Ozaki Y, et al. Relationship between coronary plaque morphology of the left anterior descending artery and 12 months clinical outcome: the CLIMA study. Eur Heart J. (2019) 41:383–91. doi: 10.1093/eurheartj/ehz520

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Waksman R, Di Mario C, Torguson R, Ali ZA, Singh V, Skinner WH, et al. Identification of patients and plaques vulnerable to future coronary events with near-infrared spectroscopy intravascular ultrasound imaging: a prospective, cohort study. Lancet. (2019) 394:1629–37. doi: 10.1016/S0140-6736(19)31794-5

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Stefanadis C, Antoniou C-K, Tsiachris D, Pietri P. Coronary atherosclerotic vulnerable plaque: current perspectives. J Am Heart Assoc. (2017) 6:e005543. doi: 10.1161/JAHA.117.005543

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Pathan F, Negishi K. Prediction of cardiovascular outcomes by imaging coronary atherosclerosis. Cardiovasc Diagn Ther. (2016) 6:322–39. doi: 10.21037/cdt.2015.12.08

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Agatston AS, Janowitz WR, Hildner FJ, Zusmer NR, Viamonte M Jr, Detrano R. Quantification of coronary artery calcium using ultrafast computed tomography. J Am Coll Cardiol. (1990) 15:827–32. doi: 10.1016/0735-1097(90)90282-T

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Otsuka F, Sakakura K, Yahagi K, Joner M, Virmani R. Has our understanding of calcification in human coronary atherosclerosis progressed? Arterioscler Thromb Vasc Biol. (2014) 34:724–36. doi: 10.1161/ATVBAHA.113.302642

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Mortensen MB, Dzaye O, Steffensen FH, Bøtker HE, Jensen JM, Rønnow Sand NP, et al. Impact of plaque burden versus stenosis on ischemic events in patients with coronary atherosclerosis. J Am Coll Cardiol. (2020) 76:2803–13. doi: 10.1016/j.jacc.2020.10.021

PubMed Abstract | CrossRef Full Text | Google Scholar