Xiaoxi Huang1*†
Haomin Yang- 1Department of Breast, Fujian Maternity and Child Health Hospital, College of Clinical Medicine for Obstetrics and Gynecology and Pediatrics, Fujian Medical University, Fuzhou, Fujian, China
- 2Department of Surgery, Changting Maternity and Children's Hospital, Longyan, Fujian, China
- 3Department of Gynecology, Changting Maternity and Children's Hospital, Longyan, Fujian, China
- 4Department of Anesthesiology, Changting Maternity and Children's Hospital, Longyan, Fujian, China
- 5Department of Public Health, Changting Maternity and Children's Hospital, Longyan, Fujian, China
- 6Department of Ultrasound, Fujian Maternity and Child Health Hospital, College of Clinical Medicine for Obstetrics and Gynecology and Pediatrics, Fujian Medical University, Fuzhou, Fujian, China
- 7Department of Epidemiology and Health Statistics, School of Public Health, Fujian Medical University, Fuzhou, Fujian, China
Background: Artificial intelligence breast ultrasound diagnostic system (AIBUS) has been introduced as an alternative approach for handheld ultrasound (HHUS), while their results in BI-RADS categorization has not been compared.
Methods: This pilot study was based on a screening program conducted from May 2020 to October 2020 in southeast China. All the participants who received both HHUS and AIBUS were included in the study (N = 344). The ultrasound videos after AIBUS scanning were independently watched by a senior radiologist and a junior radiologist. Agreement rate and weighted Kappa value were used to compare their results in BI-RADS categorization with HHUS.
Results: The detection rate of breast nodules by HHUS was 14.83%, while the detection rates were 34.01% for AIBUS videos watched by a senior radiologist and 35.76% when watched by a junior radiologist. After AIBUS scanning, the weighted Kappa value for BI-RADS categorization between videos watched by senior radiologists and HHUS was 0.497 (p < 0.001) with an agreement rate of 78.8%, indicating its potential use in breast cancer screening. However, the Kappa value of AIBUS videos watched by junior radiologist was 0.39, when comparing to HHUS.
Conclusion: AIBUS breast scan can obtain relatively clear images and detect more breast nodules. The results of AIBUS scanning watched by senior radiologists are moderately consistent with HHUS and might be used in screening practice, especially in primary health care with limited numbers of radiologists.
Introduction
Breast cancer is the most common malignant tumor in women worldwide, and the incidence is increasing every year (1). As only a small fraction of breast cancer can be attributed to the known life-style risk factor (2), early detection is key to improving survival and reducing mortality associated with breast cancer (3). Mammography is widely used for breast cancer screening, whereas ultrasound is used as a supplementary screening method for women with dense breasts in European and American countries (4). Ultrasound is also recommended for breast cancer screening in primary health care in China, especially in the rural areas (5), because of the absence of ionizing radiation, its cost-effectiveness, and a lower rate of missed diagnosis with dense breasts (6).
However, handheld ultrasound (HHUS) requires working experience of the radiologists, and it is difficult to standardize (7). Therefore, several artificial intelligence methods have been applied in breast ultrasound (8). The Artificial Intelligence Breast Ultrasound Diagnostic System (AIBUS) was developed in 2018 and used in the breast cancer screening programs of several provinces in China since 2019, with a similar detection rate of breast nodules and shorter examination time as compared to HHUS (9, 10). AIBUS can automate and standardize breast scanning and video acquisition procedures without a radiologist. The ultrasound videos are transmitted to the server immediately, and an artificial intelligence algorism is used to provide suspected lesion videos for multiple users and multiple terminals to make the diagnosis. It can also longitudinally store the ultrasound video and provide long-term breast health surveillance for women. The mobility of AIBUS may further increase the accessibility of the breast cancer screening program and address the shortage of radiologists and equipment in the primary care settings in rural and remote areas.
Despite the various comparative studies on other operator-independent automated breast scanners and HHUS (11), the agreement of Breast Imaging Reporting and Data System (BI-RADS) score categorization between AIBUS and HHUS has not been evaluated. In addition, as radiologists finally watched the videos from AIBUS and decided the BI-RADS score, it is unclear whether radiologists' experience might influence the agreement between AIBUS and HHUS.
This study aimed to analyse the agreement between AIBUS and HHUS among women who underwent both examinations and to determine the effect of radiologists' experience on categorizing BI-RADS scores.
Methods
Study population
We recruited women who participated in the breast cancer screening program in Changting in southeast China between May 2020 and October 2020. As required by the breast cancer screening program in rural China, ultrasound was the primary screening approach and these women had not received any breast examination for at least 3 years. We included those who successively underwent both HHUS and AIBUS examination, resulting in a total of 344 women aged 35–63 years old. The study was approved by the ethics committee in Fujian Maternity and Child Health Hospital and all methods were performed in accordance with relevant guidelines and regulations.
Ultrasound examination
In HHUS examination, a commercial color Doppler ultrasound equipment (Shenzhen Mindray Bio-Medical Electronics Co., Ltd, https://www.mindray.com/en/) was used with a linear transducer (≥7.5 MHz). A comprehensive scan of the breasts was performed by senior radiologists with more than 10 years of experience to observe the position, size, shape, boundary, and internal echo of the lesions.
AIBUS was performed after HHUS using the AIBUS100 Pro artificial intelligence breast ultrasound diagnostic system (Shenzhen Hanwei Intelligent Medical Technology Co., Ltd. http://www.aisono.com/). It integrates artificial intelligence, computer visualization, and robotics, with the principle based on traditional ultrasound. The women were asked to lie on their backs, and robotic arms scanned both breasts. In the scanning process, structured light, and image recognition algorithms were first used to identify scanning areas. A trajectory planning algorithm was further used to optimize the scanning route. The scanning procedure does not require a radiologist to operate on-site, realizing fully automated standardized breast scanning. After scanning, the ultrasound videos were stored and remotely watched by a senior radiologist and a junior radiologist from other hospitals. The AIBUS system also provided indications for videos with suspected nodules.
Results of HHUS and AIBUS were classified according to ultrasound BI-RADS, developed by the American College of Radiology (12). Generally, women without breast nodules are categorized as BI-RADS level 1, while the nodules are further categorized with level 2–5 according to the characteristics of the nodules. Women with BI-RADS 2 and 3 were suggested to have a regular check-up, whereas those with BI-RADS 4 and 5 were suggested for a further biopsy.
Statistical analysis
We used McNemar's chi-square test to compare the detection rate of breast nodules using HHUS and AIBUS. Kappa test with linear weighting was further used to compare the agreement of the BI-RADS scores from AIBUS and HHUS. SPSS25.0 statistical software was used for statistical analysis, and P < 0.05 indicates a statistically significant finding.
Results
The mean age of 344 participants was 47.9 years old (SD = 6.4). Majority of the women were premenopausal (65.99%) and only one woman had received breast ultrasound previously (Table 1). Among them, 51 women were detected with breast nodules using HHUS, corresponding to a detection rate of 14.83%. After watching the AIBUS scanning videos, the senior radiologists diagnosed 117 breast nodules, while the junior radiologists diagnosed 123 nodules, corresponding to the detection rates of 34.01 and 35.76%. The detection rate of breast nodules using AIBUS were significantly higher than HHUS, which detected 20% more breast nodules (Table 2).
Table 1. Basic characteristics of the study population.
Table 2. Detection rates of breast nodules using HHUS and AIBUS.
When categorized by BI-RADS scores, 293 women were classified into BI-RADS level 1, 32 into BI-RADS level 2, and 19 into BI-RADS level 3 using HHUS. After AIBUS scanning, 227 women were classified into BI-RADS level 1, 67 into BI-RADS level 2, and 50 into BI-RADS level 3, according to the videos watched by the senior radiologists. The observed agreement rate between HHUS and AIBUS was 78.8%, and the weighted Kappa value was 0.497 (P < 0.001; Table 3).
Table 3. Comparison between AIBUS read by senior radiologists and HHUS in BI-RADS categorization.
For the same videos watched by junior radiologists, 221 women were classified into BI-RADS level 1, 52 into BI-RADS level 2, and 71 into BI-RADS level 3. The corresponding agreement rate and weighted Kappa value were 74.4% and 0.390, respectively (P < 0.001; Table 4), which were lower than the Kappa value estimated from the senior radiologists.
Table 4. Comparison between AIBUS read by junior radiologists and HHUS in BI-RADS categorization.
Discussion
Key results
In this head to head comparison study, we found a significantly higher detection rate of breast nodules using AIBUS as compared to HHUS. We also observed a moderate agreement in BI-RADS categorization between HHUS and AIBUS videos watched by senior radiologists, whereas the agreement was fair when junior radiologists watched the videos.
Our analysis also showed that 22.5% (66/293) of the women without nodules detected by HHUS were categorized as BI-RADS 2 or 3 by AIBUS, suggesting that AIBUS might detect some breast nodules missed by HHUS. The missed breast nodules by HHUS had similar width and echo pattern with the concordant nodules, while some of the missed nodules did not have clear margins. As the radiologists were more focused at watching the videos recorded by AIBUS and AIBUS provided the suspected nodules videos, it is plausible that HHUS might have missed some nodules when moving the probe.
The moderate agreement between HHUS and AIBUS diagnosed by senior radiologists was similar to other operator-independent automated breast scanners (11, 13, 14), indicating that AIBUS could provide clear images for breast cancer screening. In addition, AIBUS also had a shorter examination time than the traditional HHUS (in average 222.6 s for AIBUS and 236.1 s for HHUS) (9). For AIBUS, a nurse or health worker is required only to initiate the scan, and the machine then proceeds automatically with one click. This would reduce the costs for screening, address the shortage of radiologists, and increase the accessibility of breast cancer screening, especially for women in rural and remote areas.
To our knowledge, this is the first study comparing AIBUS and HHUS in their BI-RADS categorization. As the participated women were recruited in the breast cancer screening program in China with an age limit of 35–64, findings in this study cannot be generalized to elderly women. In addition, we did not detect malignant breast nodules in these women; therefore, further studies are still needed to compare their agreement in detecting malignant tumors. However, as AIBUS detected more breast nodules than HHUS, we speculate that AIBUS would probably detect more suspicious malignant lesions than HHUS, which might increase the sensitivity of breast cancer screening.
Conclusion
In summary, we found that the results of AIBUS diagnosed by senior radiologists moderately agreed with that diagnosed by HHUS, and AIBUS detected more breast nodules. AIBUS could therefore be used in breast cancer screening in primary care setting with a shortage of local senior radiologists.
Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics statement
The studies involving human participants were reviewed and approved by the Ethics Committee in Fujian Maternity and Child Health Hospital. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.
Author contributions
XH: conceptualization, funding acquisition, visualization, and writing—original draft. YQ and XH: data curation and validation. YQ: formal analysis. YQ, FB, JWa, and CL: investigation. YQ, XH, and YL: methodology. CL: project administration. YQ, YL, and JWu: resources. YQ and JWu: software. HY and XH: supervision. YQ, FB, JWa, CL, XH, HY, YL, and JWu: writing—review and editing. All authors contributed to the article and approved the submitted version.
Funding
This work was supported by Fujian health and family planning promotion of appropriate technology projects for rural and urban communities (Grant No. 2018009).
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
1. Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, et al. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. (2021) 71:209–49. doi: 10.3322/caac.21660
2. Islami F, Chen W, Yu XQ, Lortet-Tieulent J, Zheng R, Flanders WD, et al. Cancer deaths and cases attributable to lifestyle factors and infections in China, 2013. Ann Oncol. (2017) 28:2567–74. doi: 10.1093/annonc/mdx342
3. Nattinger AB, Mitchell JL. Breast cancer screening and prevention. Ann Intern Med. (2016) 164:Itc81–96. doi: 10.7326/AITC201606070
4. Yuan W-H, Hsu H-C, Chen Y-Y, Wu C-H. Supplemental breast cancer-screening ultrasonography in women with dense breasts: a systematic review and meta-analysis. Br J Cancer. (2020) 123:673–88. doi: 10.1038/s41416-020-0928-1
5. Yang L, Wang S, Zhang L, Sheng C, Song F, Wang P, et al. Performance of ultrasonography screening for breast cancer: a systematic review and meta-analysis. BMC Cancer. (2020) 20:499. doi: 10.1186/s12885-020-06992-1
6. Berg WA, Bandos AI, Mendelson EB, Lehrer D, Jong RA, Pisano ED. Ultrasound as the primary screening test for breast cancer: analysis from ACRIN 6666. J Natl Cancer Inst. (2016) 108:djv367. doi: 10.1093/jnci/djv367
7. Yang L, Zhao Y, Du J, Chen L, He Q. Comparative study of automatic breast volume scanner and MRI in the diagnosis of breast masses. Chin J Ultrasonogr. (2017) 26:608–12. doi: 10.3760/cma.j.issn.1004-4477.2017.07.012
8. Wu GG, Zhou LQ, Xu JW, Wang JY, Wei Q, Deng YB, et al. Artificial intelligence in breast ultrasound. World J Radiol. (2019) 11:19–26. doi: 10.4329/wjr.v11.i2.19
9. Mou Y, Cao J, Jin Z, Yu X, Jiang N. Feasibility study of breast ultrasound diagnostic system applied to mass breast screening. Chin Med Equip J. (2022) 43:50–3. doi: 10.19745/j.1003-8868.2022165
10. Yu X, Cao J, Jin Z, Mou Y, Jiang N. Application value of AIBUS combined with high-frequency HHUS in population screening of breast. China Med Equip. (2022) 19:26–9. doi: 10.3969/j.issn.1672-8270.2022.05.006
11. Wang L, Qi ZH. Automatic breast volume scanner versus handheld ultrasound in differentiation of benign and malignant breast lesions: a systematic review and meta-analysis. Ultrasound Med Biol. (2019) 45:1874–81. doi: 10.1016/j.ultrasmedbio.2019.04.028
12. D'Orsi C, Morris E, Mendelson E. ACR BI-RADS RAtlas, Breast Imaging Reporting and Data System. Reston, VA: American College of Radiology (2013).
13. Yun G, Kim SM, Yun B, Ahn HS, Jang M. Reliability of automated versus handheld breast ultrasound examinations of suspicious breast masses. Ultrasonography. (2019) 38:264–71. doi: 10.14366/usg.18055
Keywords: artificial intelligence, breast ultrasound, agreement, breast cancer, cancer screening (MeSH)
Citation: Huang X, Qiu Y, Bao F, Wang J, Lin C, Lin Y, Wu J and Yang H (2023) Artificial intelligence breast ultrasound and handheld ultrasound in the BI-RADS categorization of breast lesions: A pilot head to head comparison study in screening program. Front. Public Health 10:1098639. doi: 10.3389/fpubh.2022.1098639
Received: 15 November 2022; Accepted: 30 December 2022;
Published: 18 January 2023.
Edited by:
Junjie Huang, The Chinese University of Hong Kong, ChinaCopyright © 2023 Huang, Qiu, Bao, Wang, Lin, Lin, Wu and Yang. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Xiaoxi Huang, 
5558244@qq.com
†These authors have contributed equally to this work