- 1Department of Ultrasonography, Fudan University Shanghai Cancer Center, Shanghai, China
- 2Department of Oncology, Shanghai Medical College, Fudan University, Shanghai, China
- 3Laboratory of The Smart Medicine and AI-based Radiology Technology (SMART), School of Communication and Information Engineering, Shanghai University, Shanghai, China
Purpose: The detection of human epidermal growth factor receptor 2 (HER2) expression status is essential to determining the chemotherapy regimen for breast cancer patients and to improving their prognosis. We developed a deep learning radiomics (DLR) model combining time-frequency domain features of ultrasound (US) video of breast lesions with clinical parameters for predicting HER2 expression status.
Patients and Methods: Data for this research was obtained from 807 breast cancer patients who visited from February 2019 to July 2020. Ultimately, 445 patients were included in the study. Pre-operative breast ultrasound examination videos were collected and split into a training set and a test set. Building a training set of DLR models combining time-frequency domain features and clinical features of ultrasound video of breast lesions based on the training set data to predict HER2 expression status. Test the performance of the model using test set data. The final models integrated with different classifiers are compared, and the best performing model is finally selected.
Results: The best diagnostic performance in predicting HER2 expression status is provided by an Extreme Gradient Boosting (XGBoost)-based time-frequency domain feature classifier combined with a logistic regression (LR)-based clinical parameter classifier of clinical parameters combined DLR, particularly with a high specificity of 0.917. The area under the receiver operating characteristic curve (AUC) for the test cohort was 0.810.
Conclusion: Our study provides a non-invasive imaging biomarker to predict HER2 expression status in breast cancer patients.
1 Introduction
Global cancer statistics for 2020 show that breast cancer has the highest incidence of all cancers in women and is the second leading cause of tumor-related death (1). HER2 is a transmembrane tyrosine kinase receptor whose gene is located on human chromosome 17q21 (2, 3). HER2 overexpression and gene amplification, which account for 15-20% of breast cancer patients, are important prognostic factors in breast cancer and influence the choice of therapeutic agents for breast cancer patients (4). Prior to the introduction of HER2-targeted drugs such as trastuzumab and patolizumab, HER2-positive patients had high recurrence rates and poor survival outcomes (3, 5). Trastuzumab considerably increased progression-free survival and overall survival in patients with early and advanced HER2-positive breast cancer, according to landmark trial data till 2006 (5–7).. An accurate evaluation of HER2 status is essential in the treatment of breast cancer since HER2-targeted therapy is only effective in tumors with HER2 overexpression and/or gene amplification (7–9).. Currently, in situ hybridization to assess HER2 gene amplification and immunohistochemistry (IHC) to assess protein overexpression are still the primary methods to detect HER2 expression status (10, 11). However, heterogeneity in intra-tumor HER2 expression has been reported by several research institutions, with an incidence of nearly 40% (12, 13). Preoperative core needle biopsy (CNB) can only obtain tissue from a portion of the lesion and cannot assess HER2 expression in the entire lesion. It has been shown that the concordance rate of HER2 expression status between preoperative CNB and subsequent histopathology of the resected lesion ranges from 81% to 97% (14–16). Furthermore, 20-40% of patients who receive neoadjuvant chemotherapy have altered HER2 expression (17, 18). And it has been demonstrated that HER2 intra-tumoral heterogeneity is an independent factor linked to HER2-positive patients’ insufficient response to neoadjuvant chemotherapy (19). Therefore, there is an urgent need for real-time detection of changes in HER2 expression status during neoadjuvant chemotherapy. However, in clinical practice, it is difficult for us to assess tumor HER2 expression status in real time through multiple pre-operative multi-site CNB, and pre-operative CNB are difficult to circumvent the potential bias caused by intra-tumoral heterogeneity of HER2.Therefore, it is difficult to achieve clinical real-time detection and evaluation and individualized treatment. In summary, we need to find an accurate, convenient, and non-invasive method to predict HER2 expression status to guide the individualized treatment of breast cancer patients and improve their prognosis.
Ultrasound is one of the routine preoperative examinations for breast cancer and has been widely used for the preoperative characterization of breast lesions because it is non-invasive and easy to perform. The molecular staging of breast cancer has been shown to correlate with its ultrasound characteristics (20–22). Radiomics has been widely used in the diagnosis of breast cancer (23). Although the term is not strictly defined, radiomics usually aims to extract a large amount of image information from ultrasound, CT, and other imaging images in a high-throughput manner to realize tumor segmentation, feature extraction, and model building. These features are often difficult to identify by the human eye and can be deeply correlated with the intrinsic qualities of the lesion through quantification, so radiology performs better than traditional imaging methods (24). However, the analysis of ultrasound images by radiomics still has some limitations, such as the need to manually depict the ROI to achieve tumor segmentation, which may affect the extraction of feature values (25, 26). Convolutional neural networks have had tremendous success in bioinformatics since 2012 thanks to the development of deep learning, particularly in medical imaging (27). A recent study shows that DLR can be used to analyze US images for prediction of HER2 expression status (28). However, DLR usually faces the problem of small sample learning, and this study had only 36 patients in the validation set and 108 patients in the training set. In addition, the sensitivity and specificity of the prediction model built in this study were not high, at 72.73% and 84.00%, respectively. In the process of extracting image features by DLR, it is common practice to require the sonographer to record one or several representative frames of the lesion during the examination and to perform feature extraction based on this. However, existing computer-aided classification tools tend to focus only on the final classification results and ignore the impact of key frame selection. The challenge of identifying features associated with lesions persists, along with a significant category imbalance (29).
Clinical parameters combined DLR, which integrates clinical data with network features, assists in giving information that is complementary to image features and builds models by utilizing clinical data and US image features in concert, enhancing model performance (30). It has been proposed that clinical T-stage, N-stage, and age may correlate with HER2 expression status (31). Therefore, in order to comprehensively assess HER2 expression status and improve the diagnostic performance of ultrasound prediction of HER2 expression status, we developed a clinical parameters combined DLR model based on the YOLO v5 deep neural network, which combines the time-frequency domain features of breast lesions in breast ultrasound videos with clinical parameters, and achieves both innovative 3D feature extraction of breast lesions and accurate real-time assessment of HER2 expression status.
2 Materials and methods
2.1 Patients
Data for this retrospective study were obtained from 807 patients with breast lesions who consulted the ultrasound department of the Fudan University Shanghai Cancer Center from February 2019 to July 2020. The inclusion criteria were as follows: (a) patients who underwent ultrasound within one week before surgery and whose ultrasound images showed suspected breast lesions; (b) clinical data were available; and (c) patients with the intention of undergoing breast cancer surgery. Exclusion criteria were: (a) patients who underwent CNB before ultrasonography; (b) missing primary clinical data or ultrasound video data; (c) non-compliant ultrasound video acquisition; (d) patients with multifocal lesions or bilateral disease; (e) patients with postoperatively pathologically confirmed non-invasive breast cancer and patients with benign lesions; (f) non-lumpy lesions.
2.2 Clinical characteristics
Obtained clinical and histopathological data from medical records. Histopathological findings of breast cancer include tumor type and the HER2 proliferation index. Clinical data included the patient’s menopausal status, age, US size (The transverse diameter of the largest cross-section of the lesion in ultrasonography was used as the standard), N stage, T stage, and TNM stage. According to the American Society of Clinical Oncology/College of American Pathologists Clinical Practice Guideline, patients were divided into a HER2 positive group and a HER2 negative group (10). HER2-negativity was defined as IHC 0, IHC 1+, or IHC 2+ and lack of HER2 gene amplification measured by in situ hybridization. HER2-positivity was defined as IHC 2+ or 3+ with HER2 gene amplification.
2.3 Ultrasound videos collection
Preoperative breast ultrasonography was performed by an ultrasonographer with more than 5 years of experience in breast ultrasonography, using a SuperSonic Imagine S.A, Aix-en-Provence, France color Doppler ultrasound diagnostic instrument with a selected frequency of 7~13MHz linear array transducer. During the examination, the area of the breast lesion is captured in Digital Imaging and Communications in Medicine (DICOM) format, and each frame of the video is extracted and stored in JPG format. The video acquisition method and criteria were as follows: The depth was uniformly set to 3.5 cm (including the subcutaneous fat layer and superficial pectoral muscle layer), the gain was uniformly set to 49%, and the ROI was placed in the middle of the image to obtain a standardized breast ultrasound video. The video was divided into frames for each case on the original video data to facilitate subsequent analysis.
2.4 Deep learning radiomics models
The enrolled patients were randomly divided into a training cohort and an independent test cohort in a 4:1 ratio, with the training cohort used to optimize the model parameters and the independent test cohort used for model validation (32). The multi-stage DLR-based HER2 negative-positive discrimination model proposed in this paper consists of the following analytical steps (1): A lesion detection model based on YOLO v5 neural network is trained for breast video sequences to achieve accurate localization of lesions in each frame and generate binary images of lesion regions (Figure 1) (2); 91 radiomics features were extracted from each still image frame and its mask image (3); Based on the video sequence, 24 time and frequency domain features are extracted for each static radiomics feature to form the radiomics features of the dynamic video (4); The classifier based on time-frequency domain features is integrated with the classifier of clinical variables, and the integrated model outputs the final HER2 negative-positive discrimination results (Figure 2).
Figure 1 The original breast ultrasound image and their corresponding masked image. (A) Original image; (B) Masked image.
2.4.1 Detection of breast lesions
In breast ultrasound video, the size and shape of the tumor and its distribution in the image are generally not fixed, so finding a method to accurately localize the location of the breast lesion is critical for subsequent analysis. This paper trains a breast lesion detection model based on the YOLO v5 deep neural network to achieve automatic and fast localization of lesions in ultrasound videos (33, 34).
2.4.2 Static radiomics feature extraction
The trained YOLO v5 detection model outputs the coordinates of the lesion on each breast ultrasound image frame and ensures the accuracy and coherence of the model’s detection on the overall video through subsequent rejection and interpolation processes. According to each frame of the image in Figure 1 and its corresponding mask image, its spatial domain features can be extracted for each lesion, including first-order statistical features, binary texture features, and gray-level co-generation matrix features (35).
2.4.3 Dynamic time-frequency domain feature extraction
For the dynamic characteristics of breast ultrasound video, The dynamic change curve of each static feature in time can be constructed, from which a total of 14 time-domain features can be extracted. At the same time, the fast Fourier transform of the time domain signal can obtain the frequency spectrum of the signal, and then extract a total of 10 frequency domain features, such as the direct component (dc), center of gravity frequency (fc), mean square frequency (msf).
2.5 Statistical analysis
Clinicopathological differences between the training and test sets were compared by t-test or Mann-Whitney U-test. AUC, accuracy, sensitivity, specificity, and the Youden index were used to evaluate the performance of the HER2 expression status assessment model. The AUC, accuracy, and YI differences between integrated models were compared using the paired t-test. P values less than 0.05 were regarded as statistically significant for all two-sided statistics. Statistical analysis was performed using SPSS and R software.
3 Results
3.1 Baseline characters
Between January 2019 and April 2020, 807 women with 835 breast lesions were examined; ultimately, 445 women (mean age 50 years; age range 26-83 years) with 445 malignant breast lesions were included in the study. Figure 3 shows the patient recruitment workflow. The training set contained 357 patients, while the test set contained 88 patients. Menopausal status, age, US size, N stage, T stage, and TNM stage parameters did not significantly differ between the training and test sets (Table 1). According to the results of IHC or FISH, 115 patients in the training set were HER2-positive and 242 patients were HER2-negative, and 28 patients in the validation set were HER2-positive and 60 patients were HER2-negative (Figure 4).
Figure 3 Patient recruitment workflow. In total, 835 out of 807 patients were included according to the selection criteria.
Figure 4 Patient grouping chart. A total of 445 patients. Among them, there are 357 cases in the training set and 88 cases in the test set.
3.2 Base model selection
Different base models act as classifiers and will have a significant impact on classification. To figure out the ideal base model for the HER2 prediction task, we compared the performance of Support Vector Machine (SVM), Random Forest (RF), LR and XGBoost in predicting HER2 expression status. First, the classifiers based on time-frequency domain features are constructed, and their specific statistics are shown in Table 2. The specificity of all four classifiers based on time-frequency domain features is excellent, and the classification model with RF as the feature encoder has a high specificity of 100% in the test set. Next, a classifier based on clinical variables was constructed, and its detailed statistical results are shown in Table 3. Finally, the specific statistical results of integrating the classifier based on time-frequency domain features with the classifier of clinical variables are shown in Table 4. The results showed that the prediction performance of the model integrating the time-frequency domain feature classifier and the clinical variable classifier was better than that of the classification model based on time-frequency domain features or clinical variables only. The maximum AUC value (81%) can be obtained when the time-frequency domain feature classification is based on the base model XGBoost and the clinical variables classification is based on the base model LR for model integration (Figure 5). At the same time, this model had a high specificity (91.7%). By comparing the AUC, accuracy, and YI of different integrated models through multiple replicate sampling in the test set (88 random samples in the test set and 1000 repetitions of this process to avoid differences in evaluation results due to data bias), the results all indicate that this base model combined pattern has the best performance in predicting HER2 expression status (Figure 6).
Table 4 Classification results of integrated time frequency domain feature and clinical characteristics classification model.
Figure 5 Receiver operating characteristic (ROC) curves XGBoost_LR models for predicting HER2 status. LR Logistic Regression, XGBoost Extreme Gradient Boosting.
Figure 6 Comparison of prediction performance metrics of each integrated model in 1000-time repeated validation. (A) Comparison of the Youden Index of the models. (B) Comparison of the accuracy of the models. (C) Comparison of the AUCs of the models. ACC accuracy, YI Youden index, AUC area under the receiver operating characteristic curve.
4 Discussion
For rapid real-time prediction of HER2 expression status in breast cancer patients, we developed and validated a clinical characteristics combined DLR approach based on breast ultrasound video in this study. This method showed better diagnostic performance in distinguishing HER2-negative patients from HER2-positive patients compared to models based on conventional ultrasound images or clinical characteristics only. It is encouraging that our model shows a higher specificity of 0.910 and is able to obtain a lower false-positive rate, thus reducing ineffective anti-HER2 therapy and patient harm from repeat punctures.
Intratumor heterogeneity of HER2 expression levels has been widely reported (12). On the one hand, IHC and FISH are limited by sample acquisition and can only reflect the local HER2 expression level of the tumor; on the other hand, the prevalence of FISH is limited, and this test is not available in some regions due to technical limitations, with a multicenter study showing that the acceptance rate of this test among breast cancer patients is only 6%. And because of significant differences in testing practices, test results from regional laboratories are often inaccurate compared to centralized laboratories (36). In overview, we need new methods for real-time assessment of HER2 expression status. Several investigators have suggested that HER2 expression status can be assessed using HER2 nanobodies in combination with PET-CT. However, this method is more expensive and complex, which makes it difficult to be widely used in clinical practice; furthermore, the uptake of this HER2 nanoantibody in breast in situ lesions is wide and does not differ significantly from the uptake concentration in surrounding tissues (37, 38). Other researchers attempted to develop a PET-CT radiomics mechanistic learning model to predict HER2 expression status, however, the results showed that PET-CT was not sufficient to accurately predict HER2 expression status with an AUC of 0.72-0.76 (39). Xu ZL et al. developed a deep learning model to predict HER2 expression in breast cancer from ultrasound images (28). In common with this study, our results also showed a lower overall diagnostic performance for HER2 status assessment using clinical parameters alone, with an AUC of 0.55-0.69. However, differing from that study: Firstly, this study applies the YOLOV5 deep neural network to achieve automatic detection of breast cancer lesions in breast ultrasound videos and build more complete clinical diagnostic software. This method is fast and accurate, with a detection time of 0.021 seconds for a single image, which means that it can be used in clinical practice in real time. In particular, changes in HER2 expression status can be detected in real time during neoadjuvant chemotherapy. At the same time, the deep neural network has better detection for small lesions (40). Secondly, this study introduces a new method - extracting 3D features of breast lesions from breast ultrasound videos using a deep learning method based on YOLO v5. Compared to the traditional method of taking one or a few representative frames of a breast lesion during an ultrasound scan and extracting two-dimensional features of the lesion from them. This approach allows for a more comprehensive analysis of the breast lesion while avoiding the effects of key frame selection. Finally, we combine deep learning methods with machine learning methods, i.e., we use deep learning methods for automatic tumor detection and feature extraction and different basic machine learning methods as classifiers to filter out the best performing classifier combinations. This not only improves the performance of the model but also reduces the concurrent processing bias and overfitting, which helps in pattern recognition and parameter selection (26).
The present study still has some limitations. To begin with, the experimental results of this paper show that the training set results are generally better than the test set, probably due to the use of monocentric data with more parameters and high model complexity. In addition to the use of neural network models in this study, there may be uniqueness in the representation of the hidden units of the sample data, all of which may lead to the risk of overfitting the experiment. However, it is worth noting that although the experimental results of the training set have better performance compared to the test set, the test set results are improving as the training set results keep improving. Therefore, whether there is an overfitting problem in the actual experimental process needs to be further investigated. Later, we will add external datasets, conduct multi-center experimental studies to improve the generalization performance of the model, and do more comprehensive tests. This study provides a feasible method for real-time monitoring of changes in HER2 expression status during neoadjuvant chemotherapy or targeted therapy. However, more clinical treatment data is still needed to support future applications in the clinical treatment process. Secondly, in this paper, although we have a high specificity, the sensitivity is only 64.3%. However, the sensitivity of our model has been improved compared with the prediction model based on clinical characteristics only. The sensitivity of prediction models for molecular subtypes of breast cancer based on deep learning methods in published articles ranges from 44% to 96%, and our result is within this reasonable range (41). At present, the resolution of ultrasound video is not as good as that of static 2D ultrasound images, and with future advances in imaging technology, this model is expected to achieve better diagnostic performance. Finally, in recent years, a large number of studies have found that patients with low HER2 expression (IHC 1+ or IHC 2+/in situ hybridization -) can benefit from antibody-drug conjugates (42–44). Although the development of a trichotomous predictive model of HER2 expression status is an interesting attempt, there is no radiomics model that classifies HER2 expression status more finely.
5 Conclusion
A non-invasive and practical technique for assessing HER2 expression status is provided by clinical parameters combined DLR on traditional US videos of the breast. This technique is able to identify the most suitable pharmacological therapies for breast cancer patients. Prospective multicenter validation is expected to yield additional evidence for clinical use in subsequent studies.
Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics statement
The studies involving human participants were reviewed and approved by the Ethics Committee of Cancer Hospital of Fudan University. The patients/participants provided their written informed consent to participate in this study.
Author contributions
M-YQ, Y-XH, S-CZ, and QZ designed this study. M-YQ, Y-XH, S-CZ, C-YW and QZ conducted the experiment and interpreted the data. M-YQ, C-YW, and Y-XH analyzed the data. All authors approved the final version of this manuscript.
Funding
This work was supported by the National Natural Science Foundation of China (81830058, 82071945, 92159301).
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
1. Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, et al. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin (2021) 71(3):209–49. doi: 10.3322/caac.21660
2. Hagemann IS. Molecular testing in breast cancer a guide to current practices. Arch Pathol Lab Med (2016) 140(8):815–24. doi: 10.5858/arpa.2016-0051-RA
3. Yarden Y, Sliwkowski MX. Untangling the ErbB signalling network. Nat Rev Mol Cell Biol (2001) 2(2):127–37. doi: 10.1038/35052073
4. Slamon DJ, Godolphin W, Jones LA, Holt JA, Wong SG, Keith DE, et al. Studies of the her-2/Neu proto-oncogene in human-breast and ovarian-cancer. Science. (1989) 244(4905):707–12. doi: 10.1126/science.2470152
5. Choong GM, Cullen GD, O'Sullivan CC. Evolving standards of care and new challenges in the management of HER2-positive breast cancer. CA Cancer J Clin (2020) 70(5):355–74. doi: 10.3322/caac.21634
6. Nanyan RAO, Zhimin S, Kun W. Trastuzumab plus adjuvant chemotherapy improves survival of operable HER-2 positive breast cancer. J Evidence-Based Med (2006) 6(2):74–6.
7. Slamon DJ, Leyland-Jones B, Shak S, Fuchs H, Paton V, Bajamonde A, et al. Use of chemotherapy plus a monoclonal antibody against HER2 for metastatic breast cancer that overexpresses HER2. N Engl J Med (2001) 344(11):783–92. doi: 10.1056/nejm200103153441101
8. Vogel CL, Cobleigh MA, Tripathy D, Gutheil JC, Harris LN, Fehrenbacher L, et al. Efficacy and safety of trastuzumab as a single agent in first-line treatment of HER2-overexpressing metastatic breast cancer. J Clin Oncol (2002) 20(3):719–26. doi: 10.1200/jco.2002.20.3.719
9. Verschoor N, Deger T, Jager A, Sleijfer S, Wilting SM, Martens JWM. Validity and utility of HER2/ERBB2 copy number variation assessed in liquid biopsies from breast cancer patients: A systematic review. Cancer Treat Rev (2022) 106:102384. doi: 10.1016/j.ctrv.2022.102384
10. Wolff AC, Hammond MEH, Allison KH, Harvey BE, Mangu PB, Bartlett JMS, et al. Human epidermal growth factor receptor 2 testing in breast cancer: American society of clinical Oncology/College of American pathologists clinical practice guideline focused update. J Clin Oncol (2018) 36(20):2105–22. doi: 10.1200/jco.2018.77.8738
11. Coates AS, Winer EP, Goldhirsch A, Gelber RD, Gnant M, Piccart-Gebhart M, et al. Tailoring therapies–improving the management of early breast cancer: St gallen international expert consensus on the primary therapy of early breast cancer 2015. Ann Oncol (2015) 26(8):1533–46. doi: 10.1093/annonc/mdv221
12. Muller KE, Marotti JD, Tafe LJ. Pathologic features and clinical implications of breast cancer with HER2 intratumoral genetic heterogeneity. Am J Clin Pathol (2019) 152(1):7–16. doi: 10.1093/ajcp/aqz010
13. Allison KH, Dintzis SM, Schmidt RA. Frequency of HER2 heterogeneity by fluorescence in situ hybridization according to CAP expert panel recommendations: time for a new look at how to report heterogeneity. Am J Clin Pathol (2011) 136(6):864–71. doi: 10.1309/ajcpxtzskbrip07w
14. Meattini I, Bicchierai G, Saieva C, De Benedetto D, Desideri I, Becherini C, et al. Impact of molecular subtypes classification concordance between preoperative core needle biopsy and surgical specimen on early breast cancer management: Single-institution experience and review of published literature. Eur J Surg Oncol (2017) 43(4):642–8. doi: 10.1016/j.ejso.2016.10.025
15. Seferina SC, Nap M, van den Berkmortel F, Wals J, Voogd AC, Tjan-Heijnen VCG. Reliability of receptor assessment on core needle biopsy in breast cancer patients. Tumor Biol (2013) 34(2):987–94. doi: 10.1007/s13277-012-0635-5
16. Tamaki K, Sasano H, Ishida T, Miyashita M, Takeda M, Amari M, et al. Comparison of core needle biopsy (CNB) and surgical specimens for accurate preoperative evaluation of ER, PgR and HER2 status of breast cancer patients. Cancer Sci (2010) 101(9):2074–9. doi: 10.1111/j.1349-7006.2010.01630
17. Niikura N, Tomotaki A, Miyata H, Iwamoto T, Kawai M, Anan K, et al. Changes in tumor expression of HER2 and hormone receptors status after neoadjuvant chemotherapy in 21,755 patients from the Japanese breast cancer registry. Ann Oncol (2016) 27(3):480–7. doi: 10.1093/annonc/mdv611
18. Lee HJ, Kim JY, Park SY, Park IA, Song IH, Yu JH, et al. Clinicopathologic significance of the intratumoral heterogeneity of HER2 gene amplification in HER2-positive breast cancer patients treated with adjuvant trastuzumab. Am J Clin Pathol (2015) 144(4):570–8. doi: 10.1309/ajcp51hcgpopwscy
19. Hou Y, Nitta H, Wei L, Banks PM, Portier B, Parwani AV, et al. HER2 intratumoral heterogeneity is independently associated with incomplete response to anti-HER2 neoadjuvant chemotherapy in HER2-positive breast carcinoma. Breast Cancer Res Treat (2017) 166(2):447–57. doi: 10.1007/s10549-017-4453-8
20. Sechel G, Rogozea LM, Roman NA, Ciurescu D, Cocuz ME, Manea RM. Analysis of breast cancer subtypes and their correlations with receptors and ultrasound. Rom J Morphol Embryol. (2021) 62(1):269–78. doi: 10.47162/rjme.62.1.28
21. Huang ZF, Chen L, Wang Y, Fu LN, Lv RH. Molecular markers, pathology, and ultrasound features of invasive breast cancer. Clin Imaging. (2021) 79:85–93. doi: 10.1016/j.clinimag.2021.03.039
22. Gumowska M, Mączewska J, Prostko P, Roszkowska-Purska K, Dobruch-Sobczak K. Is there a correlation between multiparametric assessment in ultrasound and intrinsic subtype of breast cancer? J Clin Med (2021) 10(22):5394. doi: 10.3390/jcm10225394
23. Conti A, Duggento A, Indovina I, Guerrisi M, Toschi N. Radiomics in breast cancer classification and prediction. Semin Cancer Biol (2021) 72:238–50. doi: 10.1016/j.semcancer.2020.04.002
25. Wang K, Lu X, Zhou H, Gao YY, Zheng J, Tong MH, et al. Deep learning radiomics of shear wave elastography significantly improved diagnostic performance for assessing liver fibrosis in chronic hepatitis b: a prospective multicentre study. Gut. (2019) 68(4):729–41. doi: 10.1136/gutjnl-2018-316204
26. Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Commun ACM. (2017) 60(6):84–90. doi: 10.1145/3065386
27. Nemade V, Pathak S, Dubey AK. A systematic literature review of breast cancer diagnosis using machine intelligence techniques. Arch Comput Methods Eng (2022) 29(6):4401–30. doi: 10.1007/s11831-022-09738-3
28. Xu ZL, Yang QW, Li MH, Gu JB, Du CP, Chen Y, et al. Predicting HER2 status in breast cancer on ultrasound images using deep learning method. Front Oncol (2022) 12:829041. doi: 10.3389/fonc.2022.829041
29. Huang R, Ying Q, Lin Z, Zheng Z, Tan L, Tang G, et al. Extracting keyframes of breast ultrasound video using deep reinforcement learning. Med Image Anal (2022) 80:102490. doi: 10.1016/j.media.2022.102490
30. Xie YT, Zhang JP, Xia Y, Fulham M, Zhang YN. Fusing texture, shape and deep model-learned information at decision level for automated classification of lung nodules on chest CT. Inf Fusion. (2018) 42:102–10. doi: 10.1016/j.inffus.2017.10.005
31. Zhang L, Li J, Xiao Y, Cui H, Du GQ, Wang Y, et al. Identifying ultrasound and clinical features of breast cancer molecular subtypes by ensemble decision. Sci Rep (2015) 5:11085. doi: 10.1038/srep11085
32. van Timmeren JE, Cester D, Tanadini-Lang S, Alkadhi H, Baessler B. Radiomics in medical imaging-"how-to" guide and critical reflection. Insights Imaging. (2020) 11(1):91. doi: 10.1186/s13244-020-00887-2
33. Aly GH, Marey M, El-Sayed SA, Tolba MF. YOLO based breast masses detection and classification in full-field digital mammograms. Comput Methods Programs Biomed (2021) 200:105823. doi: 10.1016/j.cmpb.2020.105823
34. Al-Antari MA, Han SM, Kim TS. Evaluation of deep learning detection and classification towards computer-aided diagnosis of breast lesions in digital X-ray mammograms. Comput Methods Programs Biomed (2020) 196:105584. doi: 10.1016/j.cmpb.2020.105584
35. Zhu G, Zhang Z, Wang J, Wu Y, Lu H. Dynamic collaborative tracking. IEEE Trans Neural Netw Learn Syst (2019) 30(10):3035–46. doi: 10.1109/tnnls.2018.2861838
36. Fitzgibbons P, Washington MK, Murphy D. Clinical practice patterns and cost-effectiveness of human epidermal growth receptor 2 testing strategies in breast cancer patients. Cancer. (2010) 116(16):3980–1. doi: 10.1002/cncr.25186
37. Keyaerts M, Xavier C, Heemskerk J, Devoogdt N, Everaert H, Ackaert C, et al. Phase I study of Ga-68-HER2-Nanobody for PET/CT assessment of HER2 expression in breast carcinoma. J Nucl Med (2016) 57(1):27–33. doi: 10.2967/jnumed.115.162024
38. Zhao LZ, Liu CC, Xing Y, He J, O'Doherty J, Huang WH, et al. Development of a Tc-99m-Labeled single-domain antibody for SPECT/CT assessment of HER2 expression in breast cancer. Mol Pharm (2021) 18(9):3616–22. doi: 10.1021/acs.molpharmaceut.1c00569
39. Chen Y, Wang Z, Yin G, Sui C, Liu Z, Li X, et al. Prediction of HER2 expression in breast cancer by combining PET/CT radiomic analysis and machine learning. Ann Nucl Med (2022) 36(2):172–82. doi: 10.1007/s12149-021-01688-3
40. Wan J, Chen B, Yu Y. Polyp detection from colorectum images by using attentive YOLOv5. Diagnostics (Basel). (2021) 11(12):2264. doi: 10.3390/diagnostics11122264
41. Zhao X, Bai JW, Guo Q, Ren K, Zhang GJ. Clinical applications of deep learning in breast MRI. Biochim Biophys Acta Rev Cancer. (2023) 1878(2):188864. doi: 10.1016/j.bbcan.2023.188864
42. Marchiò C, Annaratone L, Marques A, Casorzo L, Berrino E, Sapino A. Evolving concepts in HER2 evaluation in breast cancer: Heterogeneity, HER2-low carcinomas and beyond. Semin Cancer Biol (2021) 72:123–35. doi: 10.1016/j.semcancer.2020.02.016
43. Modi S, Park H, Murthy RK, Iwata H, Tamura K, Tsurutani J, et al. Antitumor activity and safety of trastuzumab deruxtecan in patients with HER2-Low-Expressing advanced breast cancer: Results from a phase ib study. J Clin Oncol (2020) 38(17):1887–96. doi: 10.1200/jco.19.02318
Keywords: ultrasound, breast cancer, human epidermal growth factor receptor 2, deep learning, YOLO V5, radiomics
Citation: Quan M-Y, Huang Y-X, Wang C-Y, Zhang Q, Chang C and Zhou S-C (2023) Deep learning radiomics model based on breast ultrasound video to predict HER2 expression status. Front. Endocrinol. 14:1144812. doi: 10.3389/fendo.2023.1144812
Received: 15 January 2023; Accepted: 27 March 2023;
Published: 18 April 2023.
Edited by:
Dong Xu, University of Chinese Academy of Sciences, ChinaReviewed by:
Wen Cheng, Harbin Medical University Cancer Hospital, ChinaLixin Jiang, Shanghai Jiao Tong University, China
Weiwei Zhan, Shanghai Jiaotong University School of Medicine, China
Copyright © 2023 Quan, Huang, Wang, Zhang, Chang and Zhou. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Shi-Chong Zhou, c2N6aG91QGhvdG1haWwuY29t; Qi Zhang, emhhbmdxQHQuc2h1LmVkdS5jbg==
†These authors have contributed equally to this work and share first authorship