Diagnostic performance of radiomics in predicting axillary lymph node metastasis in breast cancer: A systematic review and meta-analysis

Background: This study aimed to perform a meta‐analysis to evaluate the diagnostic performance of radiomics in predicting axillary lymph node metastasis (ALNM) and sentinel lymph node metastasis (SLNM) in breast cancer.

Materials and methods: Multiple electronic databases were systematically searched to identify relevant studies published before April 29, 2022: PubMed, Embase, Web of Science, Cochrane Library, China National Knowledge Infrastructure, and Wanfang Data. The quality of the included studies was assessed using the Quality Assessment of Diagnostic Accuracy Studies-2 tool. The overall diagnostic odds ratio (DOR), sensitivity, specificity, and area under the curve (AUC) were calculated to evaluate the diagnostic performance of radiomic features for lymph node metastasis (LNM) in patients with breast cancer. Spearman’s correlation coefficient was determined to assess the threshold effect, and meta-regression and subgroup analyses were performed to explore the possible causes of heterogeneity.

Results: A total of 30 studies with 5611 patients were included in the meta-analysis. Pooled estimates suggesting overall diagnostic accuracy of radiomics in detecting LNM were determined: DOR, 23 (95% CI, 16-33); sensitivity, 0.86 (95% CI, 0.82-0.88); specificity, 0.79 (95% CI, 0.73-0.84); and AUC, 0.90 (95% CI, 0.87-0.92). The meta-analysis showed significant heterogeneity between sensitivity and specificity across the included studies, with no evidence for a threshold effect. Meta-regression and subgroup analyses showed that combined clinical factors, modeling method, region, and imaging modality (magnetic resonance imaging [MRI], ultrasound, computed tomography [CT], and X-ray mammography [MMG]) contributed to the heterogeneity in the sensitivity analysis (P < 0.05). Furthermore, modeling methods, MRI, and MMG contributed to the heterogeneity in the specificity analysis (P < 0.05).

Conclusion: Our results show that radiomics has good diagnostic performance in predicting ALNM and SLNM in breast cancer. Thus, we propose this approach as a clinical method for the preoperative identification of LNM.

Introduction

Female breast cancer has recently surpassed lung cancer to become the most common cancer worldwide (1). Increasingly young patients develop breast cancer, among which axillary lymph node metastasis (ALNM) is the most common mode of metastasis. Axillary lymph nodes (ALN) reside in the axillary region of the upper limb and carry important information for the anatomic staging of breast cancer. The presence and extent of axillary lymph node metastasis reflect the risk of distant recurrence and death after topical treatment. Moreover, the overall survival rate of node-positive patients is 40% lower than node-negative (2, 3). Thus, the lymph node status is crucial for treatment planning, surgical procedures, and prognosis assessment of patients (4).Currently, staging of the axilla in patients with breast cancer is evaluated with sentinel lymph node biopsy and axillary lymph node dissection. However, both involve invasive surgery and have a high incidence of complications (5–7). Therefore, developing noninvasive ALN staging methods for determining the clinical stage of breast cancer and selecting individualized treatment options for patients is paramount.

Radiomics is a novel noninvasive method that rapidly extracts numerous quantitative features imperceptible to the naked eye from traditional biomedical images through high-throughput computation. It provides valuable diagnostic and prognostic information by analyzing the correlation between image features and clinical data, which is widely used in tumor grading, therapeutic response, and prognostic assessment (8, 9).In breast cancer, radiomics constructs predictive models for ALNM and SLNM by extracting quantitative features from ultrasound, computed tomography (CT), magnetic resonance imaging (MRI), X-ray mammography (MMG), or positron emission tomography (PET), with good potential (10–14). However, due to differences in study methods and imaging modalities, heterogeneity may arise among the studies. Moreover, in patients with breast cancer, systematic research on the performance of radiomics in predicting lymph node metastasis (LNM) and the factors affecting it is lacking, necessitating further verification of radiomics for clinical application.

Therefore, the objective of this systematic review with meta-analysis was to evaluate the diagnostic performance of radiomics models in predicting ALNM and SLNM in patients with breast cancer.

Materials and methods

The meta-analysis was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) Statement (15).

Literature search

Two observers independently searched various electronic databases: PubMed, Embase, Web of Science, Cochrane Library, China National Knowledge Infrastructure, and Wanfang Data to identify eligible studies published before April 29, 2022. “Breast Neoplasms,” “Lymphatic Metastasis,” “Radiomics,” and their variations were used as search terms. Only English and Chinese publications were sought, and EndNote 20 software was used for reference management. Disagreement among the observers, if any, was resolved with a consensus achieved through discussion with the third observer.

Selection criteria

The inclusion criteria were as follows:

1. Original research studies

2. Studies on patients with breast cancer and ALNM or SLNM, according to pathological criteria

3. Radiomics studies, including ultrasound, CT, MRI, MMG, or PET–CT images applied for ALNM or SLNM classification

4. Data sufficient to reconstruct the 2 × 2 contingency table to estimate the sensitivity and specificity of the diagnosis

The exclusion criteria were as follows:

1. Reviews, editorials, expert opinions, animal studies, and conference presentations

2. Imaging analysis based on only non-radiomics methods

3. Unclear pathological diagnosis

4. Studies with smaller sample sizes, if involving overlapping patients and data

Data extraction

Relevant data were extracted from the included studies: the first author, publication year, study design, country, sample size, the number of LNM and non-LNM in breast cancer, reference standard, imaging modalities, radiomics algorithm, and clinical factors. True-positive, false-positive, false-negative, and true-negative values were extracted from the data, and the 2×2 contingency table was generated. When multiple models were used for a patient group, the one with the highest diagnostic accuracy was selected for the meta-analysis. Disagreement was discussed until consensus was reached.

Data quality assessment

The risk of bias in the selected studies was assessed using the 4 areas of the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool: patient selection, index test, reference standard, and flow and timing, customized to a particular study question (16).

Statistical analysis

Statistics were calculated using the Midas modules in the Stata software (v 16.0) and Review Manager (v 5.3). The pooled sensitivity, specificity, diagnostic odds ratio (DOR), positive and negative likelihood ratios (LR), with corresponding 95% confidence intervals (CIs), were determined to predict diagnostic accuracy. The summary receiver operating characteristic (SROC) curve and the area under the curve (AUC) were also constructed using the random effects model to evaluate the diagnostic value of combined studies (17). The AUC values suggested the discriminatory power as follows: low, 0.5-0.7; moderate, 0.7-0.9; and high, > 0.9.

Heterogeneity between the included studies was assessed with a Cochran Q test and the I² statistics. A difference was considered significant when P < 0.05, with I² ≥ 50% indicating a moderate-to-high heterogeneity among the studies. Pooling of studies and effect size were evaluated using a random-effect model, suggesting the distribution of true effects among the heterogenous studies (18). The causes of heterogeneity were estimated with threshold effect and meta-regression analyses. Spearman’s correlation coefficient was calculated to assess the threshold heterogeneity, with P < 0.05 implying a threshold effect (19). Univariable meta-regression analysis of several relevant covariates: study design (retrospective or prospective), combined clinical factors (yes or no), modeling method (radiomic algorithm or machine/deep learning), region (China or other), and imaging modality (MRI, ultrasound, CT, PET–CT or MMG), was performed.

In addition, to assess the relative contribution of a single study to the overall estimate, sensitivity analyses were done by sequential exclusion of one study from the meta-analysis calculations. Publication bias was examined using Deek’s funnel plots, where slope coefficients with P < 0.10 indicated significant publication bias (20).

Clinical utility

The clinical utility of radiomics in predicting LNM was evaluated with Fagan plot analysis by indicating the posttest probability (P post) of LNM when pretest probabilities (P pre, suspicion of LNM) were provided (21).

Results

Literature search

According to search strategies described in the methods section, 723 records were initially retrieved from the electronic databases. After removing 114 duplicates, 609 studies remained. They were screened by title and abstract for relevance, and 536 were excluded based on the inclusion and exclusion criteria. Finally, the 2 independent observers scrutinized 73 full-text articles and included 30 in the meta-analysis. Figure 1 shows the flow chart that summarizes the study selection process.

FIGURE 1

Figure 1 Flow diagram of study selection for meta-analysis according to PRISMA. PRISMA, Preferred Reported Items for Systematic Reviews and Meta analyses.

Data quality assessment

Overall, most studies exhibited very low risk of bias and few applicability problems, suggesting high data quality. The details of the risk of bias and applicability concerns of the included studies are presented in Figure 2.

FIGURE 2

Figure 2 Quality assessment of included studies according to Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) criteria. (A) Individual studies, (B) summary. Green, yellow, and red circles denote low, unclear, and high bias risks, respectively.

Characteristics of the included studies

Table 1 shows relevant characteristics and details of 30 studies included in the meta-analysis. The studies had 5611 patients, 2337 with LNM and 3274 without, assessed using a radiomics method. Seven studies (3, 22–27) reported diagnostic performance based on radiological characteristics and clinical factors. Only one (10) used prospectively collected data. Furthermore, the studies applied radiomics based on different imaging modalities: MRI, 17 (3, 11, 22, 23, 28–39); ultrasound, 7 (14, 24, 25, 27, 40–42); CT, 2 (12, 43); PET–CT, 2 (10, 44); and MMG, 3 (13, 26, 45).

TABLE 1

Table 1 The baseline characteristics of included studies.

Data analysis

Pooled estimates reflecting the overall predictive value of radiomic signatures for ALNM and SLNM in breast cancer were calculated across the studies: sensitivity, 0.86 (95% CI, 0.82-0.88); specificity, 0.79 (95% CI, 0.73-0.84); positive LR, 4.2 (95% CI, 3.2-5.4); negative LR, 0.18 (95% CI, 0.15-0.22); and DOR, 23 (95% CI, 16-33). In addition, an SROC curve was generated, and the AUC of the SROC curve was 0.90 (95% CI, 0.87-0.92), indicating a high overall diagnostic value. The forest plots for sensitivity and specificity are illustrated in Figure 3, and the SROC curve is presented in Figure 4.

FIGURE 3

Figure 3 Forest plots of the sensitivity and specificity of radiomics in predicting ALNM in breast cancer. Numbers are pooled estimates, with 95% CIs depicted with horizontal lines. Heterogeneity statistics are shown at bottom right. I²>50% indicates substantial heterogeneity in the diagnostic parameters across studies.

FIGURE 4

Figure 4 The summary receiver operating characteristic (SROC) curve of the diagnostic accuracy of radiomics in predicting ALNM in breast cancer. Each circle indicates one included study. Values in brackets are 95% CIs. AUC, area under the receiver operating characteristic curve.

Heterogeneity estimation

The I² statistic revealed significant heterogeneity between sensitivity (I² = 79.69%) and specificity (I² = 93.75%) across the studies. Spearman’s correlation coefficient of 0.185 (P = 0.328) was calculated, implying the threshold effect does not exist.

Next, univariable meta-regression and subgroup analyses were performed to identify the source of heterogeneity. Table 2 summarizes the univariable meta-regression and subgroup analyses results, suggesting the influence of study design, clinical factors, modeling methods, region, and specific imaging modality on the pooled sensitivity and specificity estimates.

TABLE 2

Table 2 Univariable meta-regression and subgroup analyses.

In terms of study design, retrospective studies (n = 29) had higher sensitivity 0.86 (95% CI, 0.83-0.89) and specificity 0.80 (95% CI, 0.74-0.85) than the prospective (n = 1; sensitivity, 0.80 [95% CI, 0.61-0.99]; specificity, 0.69 [95% CI, 0.31-1.00.]). The pooled sensitivity 0.87 (95% CI, 0.82-0.93) and specificity 0.84 (95% CI, 0.75-0.93) of radiomics combined with clinical factors (n = 7) were slightly higher than those using only radiomic features (n = 23; sensitivity, 0.85 [95% CI, 0.82-0.89]; specificity, 0.78 (95% CI, 0.71-0.84]). If a modeling method used radiomic algorithm or machine/deep learning, either showed similar sensitivity (0.85 [95% CI, 0.81-0.90] vs 0.86 [95% CI, 0.82-0.90]) and specificity (0.79 [95% CI, 0.71-0.88] vs 0.80 [95% CI, 0.72-0.87]). Similarly, region (China or other countries) yielded almost equivalent sensitivity (0.86 [95% CI, 0.83-0.89] vs 0.84 [95% CI, 0.75-0.94]) and specificity (0.79 [95% CI, 0.74-0.85] vs 0.79 [95% CI, 0.63-0.95]). In terms of different imaging modalities, 7 studies that used ultrasound had a higher sensitivity 0.87 (95% CI, 0.81-0.92) and specificity 0.82 (95% CI, 0.72-0.92) than MRI (n = 16; sensitivity, 0.85 [95% CI, 0.81-0.90]; specificity, 0.81 [95% CI, 0.74-0.88]), PET–CT (n = 2; sensitivity, 0.83 [95% CI; 0.69-0.98]; specificity, 0.70 [95% CI, 0.42-0.99]), and MMG (n = 3; sensitivity, 0.86 [95% CI, 0.78-0.95]; specificity, 0.61 [95% CI, 0.37-0.84]). Although the pooled sensitivity 0.84 (95% CI, 0.72-0.96) of the 2 CT studies was lower than that of ultrasound, the pooled specificity 0.86 (95% CI, 0.71-1.00) was higher. The source of heterogeneity between sensitivity and specificity across the studies varies depending on the tested covariate.

Sensitivity analyses

No significant changes were observed in the overall heterogeneity among the included studies when one study was sequentially omitted from the analysis. Hence, these results agree with those of the main analyses and are shown in Table 3.

TABLE 3

Table 3 Sensitivity analysis based on radiomics in predicting ALNM in breast cancer.

Publication bias

Deek’s funnel plot revealed a high slope coefficient (P = 0.88) (Figure 5), suggesting the absence of publication bias among the studies.

FIGURE 5

Figure 5 Deeks’ funnel plot asymmetry test for publication bias. Deeks’ funnel plot shows no asymmetry and the presence of publication bias. Each circle indicates one included study. ESS, effective sample size.

Clinical utility

Using radiomic features based on various imaging modalities would increase the posttest probability to 51% from 20% with a PLR of 4 when the pretest was positive and would reduce the posttest probability to 4% with an NLR of 0.18 when the pretest was negative (Figure 6).

FIGURE 6

Figure 6 Fagan plots for assessing the clinical utility. LR, likelihood ratio; Prob, probability; Pos, positive; Neg, negative.

Radiomic features based on various imaging modalities increased the posttest probability to 51% from 20%, with a positive LR of 4 when the pretest was positive. Conversely, they reduced the posttest probability to 4%, with a negative LR of 0.18 when the pretest was negative (Figure 6).

Discussion

In our systematic review with meta-analysis, we determined pooled estimates of sensitivity 0.86 (95% CI, 0.82-0.88), specificity 0.79 (95% CI, 0.73-0.84), and AUC 0.90 (95% CI, 0.87-0.92). These values indicate that radiomics is an effective and accurate tool for predicting ALNM and SLNM in breast cancer, which will help clinicians select safe and effective treatments for patients and reduce postoperative complications.

Sensitivity analyses were performed on the 30 included studies with a sequential exclusion approach. Despite omitting a study with each step, pooled sensitivity and specificity were unaffected, and radiomics continued to show good predictive ability for ALNM and SLNM. Furthermore, Deek’s funnel plot pointed to the absence of published bias, suggesting our results are reliable. Likelihood ratios and posttest probabilities are indexes of the diagnostic tests and provide crucial information about the likelihood that a patient with a positive or negative test has LNM. Radiomic features increased the posttest probability to 51% from 20%, with a positive LR of 4 when the pretest was positive. By contrast, the signatures reduced the posttest probability to 4%, with a negative LR of 0.18 when the pretest was negative. These data demonstrate that radiomics has important clinical value in improving the accuracy of predicting LNM in patients with breast cancer.

Heterogeneity is common in meta-analysis. In our study, we observed significant heterogeneity between the studies, which could have been caused by threshold effect, study design, or even imaging modality. Spearman’s correlation coefficient value was insignificant, eliminating the threshold effect as the potential source of heterogeneity in our meta-analysis. Therefore, meta-regression and subgroup analyses were performed to identify the source of heterogeneity, but due to the limited number of included studies, univariate meta-regression analyses were performed instead of multivariate meta-regression analyses. Differences in study design and region can provide different information and research quality and cause heterogeneity. Various clinical factors combined with radiomics confer high diagnostic efficiency. However, they can also be a source of heterogeneity. Furthermore, different modeling methods and imaging modalities also create heterogeneity (10–14). Although previous studies considered different sources of heterogeneity, a systematic study of the radiomics performance in predicting ALNM and SLNM in breast cancer was lacking. Therefore, we used 5 parameters for univariate meta-regression and subgroup analysis to identify the probable sources of heterogeneity to help address the gap.

Subgroup analysis revealed that although retrospective studies were superior to the prospective, their results were insignificant. Because most of the studies in this meta-analysis were retrospective (29 out of 30), they are prone to selection bias. Prospective studies result in better research quality due to their completeness. In addition, since exposure is evaluated before the outcome, they are less prone to bias. Therefore, future prospective studies should improve the predictive performance of radiomics for the evaluation of LNM in patients with breast cancer and its clinical efficiency. Our meta-analysis also showed that studies using radiomics combined with clinical factors have higher diagnostic performance than those relying only on radiomics, which is consistent with the previous studies (3, 25, 26). Thus, adding clinical features to radiomic imaging improves the accuracy of diagnosing LNM in breast cancer.

Subgroup analysis also uncovered that studies conducted in China had better diagnostic performance than those done abroad. However, this meta-analysis included only 4 studies from outside China, underscoring the need for more overseas studies to confirm the above conclusion. Modeling methods using radiomics algorithm and machine/deep learning had a similar diagnostic performance. This result opposes previous evidence that suggests that machine/deep learning is superior to the traditional radiomics algorithm in predicting LNM in breast cancer (3, 39, 42) and requires further verification.

Finally, we performed a subgroup analysis of the diagnostic performance of radiomics with different imaging modalities. Although ultrasound had the highest diagnostic performance, only 7 ultrasound studies were included, of which 5 combined with clinical factors or deep learning algorithms. Therefore, the pooled results do not entirely prove that ultrasound has the highest diagnostic performance. Moreover, this modality is susceptible to operator subjectivity (46). The radiomics model based on feature extraction of preoperative MRI images also had a high predictive performance. This result is similar to a previous study that used dynamic contrast-enhanced–MRI (DCE–MRI) radiomic signatures of patients with breast cancer to predict LNM with satisfactory results (47). The pooled sensitivity of MMG was similar to that of ultrasound and MRI, but pooled specificity was lower (only 0.61). Only 4 included studies combined radiomics with CT or PET–CT. Hence, the statistical differences calculated with the subgroup analysis were insignificant, and more studies with these modalities should verify their diagnostic performance. Although this meta-analysis addresses various sources of heterogeneity, other unmentioned differences between the studies may have also contributed to heterogeneity.

This study has several advantages. First, to our knowledge, it is the first meta-analysis that comprehensively evaluates the diagnostic test accuracy of radiomics models in predicting ALNM and SLNM in breast cancer. Before it, a meta-analysis (48) of DCE –MRI based on machine learning also had a good diagnostic performance, with an AUC of 0.89, but it was only a part of radiomics. Second, this study considers different imaging modalities and modeling methods, providing novel ideas for subsequent radiomics research.

We also acknowledge that our meta-analysis has certain limitations. We mainly included retrospective studies and only 1 prospective, which makes our study prone to patient selection bias and data loss. Therefore, more prospective trials are necessary to validate our findings. In addition, we found significant heterogeneity in the pooled sensitivity and specificity. This observation is similar to that in published meta-analyses of diagnostic accuracy using radiomics (49–52). Although radiomic models help predict the diagnosis of LNM in breast cancer, they involve numerous analysis methods. Thus, the choice of imaging modality and modeling method may affect the predictive results of radiomics analysis, causing heterogeneity. Furthermore, during data extraction, we selected the model with the highest diagnostic performance among multiple, which might have overestimated the pooled sensitivity and specificity of radiomics for LNM in breast cancer. Finally, most of the included studies were performed in China, which might have affected the general applicability of the results in clinical practice.

Conclusion

In conclusion, our meta-analysis demonstrates that radiomic models based on preoperative imaging features have good diagnostic performance in predicting ALNM and SLNM in patients with breast cancer. Radiomics is a promising noninvasive method and is expected to provide new quantitative diagnostic techniques for clinical practice. Future well-designed radiomics experiments are needed to verify its effectiveness and diagnostic performance, reduce its heterogeneity, and enable its wide clinical application.

Data availability statement

The original contributions presented in the study are included in the article/supplementary material. Further inquiries can be directed to the corresponding author.

Author contributions

Conception and study design: XG, YG, and MZ. Literature search and study selection: XG, YG, and TZ. Data acquisition and quality assessment: TZ and XP. Data analysis: XG, YG, TZ, and XP. Study supervision: DX and MZ. Manuscript writing: XG, YG, TZ, XP, DX, and MZ. Administrative support: DX and MZ. All authors contributed to the article and approved the submitted version.

Funding

This work was supported by Natural Science Foundation of Shanghai (Research Grant No.19ZR1452400) and National Natural Science Foundation of China (Research Grant No. 81673743).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, et al. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin (2021) 71(3):209–49. doi: 10.3322/caac.21660

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Tonellotto F, Bergmann A, de Souza Abrahão K, de Aguiar SS, Bello MA, Thuler LCS. Impact of number of positive lymph nodes and lymph node ratio on survival of women with node-positive breast cancer. Eur J Breast Health (2019) 15(2):76–84. doi: 10.5152/ejbh.2019.4414

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Cattell R, Ying J, Lei L, Ding J, Chen S, Serrano Sosa M, et al. Preoperative prediction of lymph node metastasis using deep learning-based features. Vis Comput Ind BioMed Art (2022) 5(1):8. doi: 10.1186/s42492-022-00104-5

PubMed Abstract | CrossRef Full Text | Google Scholar

4. To B, Isaac D, Andrechek ER. Studying lymphatic metastasis in breast cancer: Current models, strategies, and clinical perspectives. J Mammary Gland Biol Neoplasia (2020) 25(3):191–203. doi: 10.1007/s10911-020-09460-5

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Giuliano AE, Hunt KK, Ballman KV, Beitsch PD, Whitworth PW, Blumencranz PW, et al. Axillary dissection vs no axillary dissection in women with invasive breast cancer and sentinel node metastasis: a randomized clinical trial. Jama (2011) 305(6):569–75. doi: 10.1001/jama.2011.90

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Lyman GH, Somerfield MR, Bosserman LD, Perkins CL, Weaver DL, Giuliano AE. Sentinel lymph node biopsy for patients with early-stage breast cancer: American society of clinical oncology clinical practice guideline update. J Clin Oncol (2017) 35(5):561–4. doi: 10.1200/jco.2016.71.0947

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Lyman GH, Temin S, Edge SB, Newman LA, Turner RR, Weaver DL, et al. Sentinel lymph node biopsy for patients with early-stage breast cancer: American society of clinical oncology clinical practice guideline update. J Clin Oncol (2014) 32(13):1365–83. doi: 10.1200/jco.2013.54.1177

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Bi WL, Hosny A, Schabath MB, Giger ML, Birkbak NJ, Mehrtash A, et al. Artificial intelligence in cancer imaging: Clinical challenges and applications. CA Cancer J Clin (2019) 69(2):127–57. doi: 10.3322/caac.21552

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Gillies RJ, Kinahan PE, Hricak H. Radiomics: Images are more than pictures, they are data. Radiology (2016) 278(2):563–77. doi: 10.1148/radiol.2015151169

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Cheng JY, Ren CY, Liu GY, Shui RH, Zhang YJ, Li JJ, et al. Development of high-resolution dedicated PET-based radiomics machine learning model to predict axillary lymph node status in early-stage breast cancer. Cancers (2022) 14(4):950. doi: 10.3390/cancers14040950

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Song D, Yang F, Zhang Y, Guo Y, Qu Y, Zhang X, et al. Dynamic contrast-enhanced MRI radiomics nomogram for predicting axillary lymph node metastasis in breast cancer. Cancer Imaging (2022) 22(1):17. doi: 10.1186/s40644-022-00450-w

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Yang CM, Dong J, Liu ZY, Guo QX, Nie Y, Huang DQ, et al. Prediction of metastasis in the axillary lymph nodes of patients with breast cancer: A radiomics method based on contrast-enhanced computed tomography. Front Oncol (2021) 11:726240. doi: 10.3389/fonc.2021.726240

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Yang JB, Wang T, Yang LF, Wang YB, Li HM, Zhou XB, et al. Preoperative prediction of axillary lymph node metastasis in breast cancer using mammography-based radiomics method. Sci Rep (2019) 9(1):4429. doi: 10.1038/s41598-019-40831-z

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Zhou WJ, Zhang YD, Kong WT, Zhang CX, Zhang B. Preoperative prediction of axillary lymph node metastasis in patients with breast cancer based on radiomics of gray-scale ultrasonography. Gland Surg (2021) 10(6):1989–2001. doi: 10.21037/gs-21-315

PubMed Abstract | CrossRef Full Text | Google Scholar

15. McInnes MDF, Moher D, Thombs BD, McGrath TA, Bossuyt PM, Clifford T, et al. Preferred reporting items for a systematic review and meta-analysis of diagnostic test accuracy studies: The PRISMA-DTA statement. Jama (2018) 319(4):388–96. doi: 10.1001/jama.2017.19163

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med (2011) 155(8):529–36. doi: 10.7326/0003-4819-155-8-201110180-00009

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Moses LE, Shapiro D, Littenberg B. Combining independent studies of a diagnostic test into a summary ROC curve: data-analytic approaches and some additional considerations. Stat Med (1993) 12(14):1293–316. doi: 10.1002/sim.4780121403

PubMed Abstract | CrossRef Full Text | Google Scholar

18. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials (1986) 7(3):177–88. doi: 10.1016/0197-2456(86)90046-2

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Devillé WL, Buntinx F, Bouter LM, Montori VM, de Vet HC, van der Windt DA, et al. Conducting systematic reviews of diagnostic studies: didactic guidelines. BMC Med Res Methodol (2002) 2:9. doi: 10.1186/1471-2288-2-9

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Deeks JJ, Macaskill P, Irwig L. The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed. J Clin Epidemiol (2005) 58(9):882–93. doi: 10.1016/j.jclinepi.2005.01.016

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Hellmich M, Lehmacher W. A ruler for interpreting diagnostic test results. Methods Inf Med (2005) 44(1):124–6. doi: 10.1055/s-0038-1633930

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Chen JM, Zhu HY, Gao J, Ge YQ, Wang MH, Li Y, et al. Prediction of axillary lymph node metastasis in breast cancer based on clinical pathology and conventional and functional MRI imaging omics models Chinese medical imaging technology. Chin J Med Imaging Technol (2021) 37(06):885–90. doi: 10.13929/j.issn.1003-3289.2021.06.022

CrossRef Full Text | Google Scholar

23. Xia XD, Duan CZ, Ling M, Wang YL, Zhou XS, Wia GX, et al. Prediction of axillary lymph node metastasis in breast cancer based on MRI omics line mapping magnetic resonance imaging. Chin J Magn Reson (2022) 13(01):118–22. doi: 10.12015/issn.1674-8034.2022.01.024

CrossRef Full Text | Google Scholar

24. Jin H, Luo WQ, Ji ZP, Zhang Y, Zhao MZ, Li C, et al. Logistic regression equation predicts axillary lymph node metastasis risk in axillary lymph nodes. Chin J Ultrasound Med (2021) 37(02):139–42. doi: 10.3969/j.issn.1002-0101.2021.02.007

CrossRef Full Text | Google Scholar

25. Zha HL, Zong M, Liu XP, Pan JZ, Wang H, Gong HY, et al. Preoperative ultrasound-based radiomics score can improve the accuracy of the memorial Sloan Kettering cancer center nomogram for predicting sentinel lymph node metastasis in breast cancer. Eur J Radiol (2021) 135:109512. doi: 10.1016/j.ejrad.2020.109512

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Mao N, Yin P, Li Q, Wang Q, Liu M, Ma H, et al. Radiomics nomogram of contrast-enhanced spectral mammography for prediction of axillary lymph node metastasis in breast cancer: a multicenter study. Eur Radiol (2020) 30(12):6732–9. doi: 10.1007/s00330-020-07016-z

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Yu FH, Wang JX, Ye XH, Deng J, Hang J, Yang B. Ultrasound-based radiomics nomogram: A potential biomarker to predict axillary lymph node metastasis in early-stage invasive breast cancer. Eur J Radiol (2019) 119:108658. doi: 10.1016/j.ejrad.2019.108658

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Santucci D, Faiella E, Cordelli E, Sicilia R, de Felice C, Zobel BB, et al. 3T MRI-radiomic approach to predict for lymph node status in breast cancer patients. Cancers (2021) 13(9):2228. doi: 10.3390/cancers13092228

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Liu SH, Peng RR, Li FZ, Chen J. A study based on enhanced MRI imaging omics in breast cancer axillary lymph node metastasis predictive model Jiangxi pharmaceutical. Jiangxi Medical Journal (2021) 56(10):1615–1618+1638. doi: 10.3969/j.issn.1006-2238.2021.10.009

CrossRef Full Text | Google Scholar

30. Wang CH, Chen XY, Luo HB, Liu YY, Meng RR, Wang M, et al. Development and internal validation of a preoperative prediction model for sentinel lymph node status in breast cancer: Combining radiomics signature and clinical factors. Front Oncol (2021) 11:754843. doi: 10.3389/fonc.2021.754843

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Arefan D, Chai R, Sun M, Zuley ML, Wu S. Machine learning prediction of axillary lymph node metastasis in breast cancer: 2D versus 3D radiomic features. Med Phys (2020) 47(12):6334–42. doi: 10.1002/mp.14538

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Qiu X, Fu Y, Ye Y, Wang Z, Cao C. A nomogram based on molecular biomarkers and radiomics to predict lymph node metastasis in breast cancer. Front Oncol (2022) 12:790076. doi: 10.3389/fonc.2022.790076

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Zhan CA, Hu YQ, Wang XR, Liu H, Xia LM, Ai T. Prediction of axillary lymph node metastasis in breast cancer using intra-peritumoral textural transition analysis based on dynamic contrast-enhanced magnetic resonance imaging. Acad Radiol (2022) 29:S107–15. doi: 10.1016/j.acra.2021.02.008

CrossRef Full Text | Google Scholar

34. Cui X, Wang N, Zhao Y, Chen S, Li S, Xu M, et al. Preoperative prediction of axillary lymph node metastasis in breast cancer using radiomics features of DCE-MRI. Sci Rep (2019) 9(1):2240. doi: 10.1038/s41598-019-38502-0

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Tan H, Gan F, Wu Y, Zhou J, Tian J, Lin Y, et al. Preoperative prediction of axillary lymph node metastasis in breast carcinoma using radiomics features based on the fat-suppressed T2 sequence. Acad Radiol (2020) 27(9):1217–25. doi: 10.1016/j.acra.2019.11.004

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Liu C, Ding J, Spuhler K, Gao Y, Serrano Sosa M, Moriarty M, et al. Preoperative prediction of sentinel lymph node metastasis in breast cancer by radiomic signatures from dynamic contrast-enhanced MRI. J Magn Reson Imaging (2019) 49(1):131–40. doi: 10.1002/jmri.26224

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Han L, Zhu Y, Liu Z, Yu T, He C, Jiang W, et al. Radiomic nomogram for prediction of axillary lymph node metastasis in breast cancer. Eur Radiol (2019) 29(7):3820–9. doi: 10.1007/s00330-018-5981-2

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Liu J, Sun D, Chen L, Fang Z, Song W, Guo D, et al. Radiomics analysis of dynamic contrast-enhanced magnetic resonance imaging for the prediction of sentinel lymph node metastasis in breast cancer. Front Oncol (2019) 9:980. doi: 10.3389/fonc.2019.00980

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Zhu YD, Yang L, Shen HL. Value of the application of CE-MRI radiomics and machine learning in preoperative prediction of sentinel lymph node metastasis in breast cancer. Front Oncol (2021) 11:757111. doi: 10.3389/fonc.2021.757111

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Yu LN, Wang Y, Chen D, Liu YZ. Value of ultrasound radiomics labels in predicting sentinel lymph node metastasis in breast cancer. J Pract Med (2021) 37(15):2007–11. doi: 10.3969/j.issn.1006⁃5725.2021.15.020

CrossRef Full Text | Google Scholar

41. Guo X, Liu Z, Sun C, Zhang L, Wang Y, Li Z, et al. Deep learning radiomics of ultrasonography: Identifying the risk of axillary non-sentinel lymph node involvement in primary breast cancer. EBioMedicine (2020) 60:103018. doi: 10.1016/j.ebiom.2020.103018

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Sun Q, Lin X, Zhao Y, Li L, Yan K, Liang D, et al. Deep learning vs. radiomics for predicting axillary lymph node metastasis of breast cancer using ultrasound images: Don't forget the peritumoral region. Front Oncol (2020) 10:53. doi: 10.3389/fonc.2020.00053

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Yang XJ, Wu L, Ye WT, Zhao K, Wang YY, Liu WX, et al. Deep learning signature based on staging CT for preoperative prediction of sentinel lymph node metastasis in breast cancer. Acad Radiol (2020) 27(9):1226–33. doi: 10.1016/j.acra.2019.11.007

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Song B-I. A machine learning-based radiomics model for the prediction of axillary lymph-node metastasis in breast cancer. Breast Cancer (2021) 28(3):664–71. doi: 10.1007/s12282-020-01202-z

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Zhang YJ, Song DL, Wang YF, Ma YQ, Yang F, Zhu YX, et al. Predictive value of imaging omics line line based on mammography for axillary lymph node metastasis in breast cancer radiology in practice. Radiol Practice (2022) 37(01):48–54. doi: 10.13609/j.cnki.1000-0313.2022.01.009

CrossRef Full Text | Google Scholar

46. Choi HY, Park M, Seo M, Song E, Shin SY, Sohn YM. Preoperative axillary lymph node evaluation in breast cancer: Current issues and literature review. Ultrasound Q (2017) 33(1):6–14. doi: 10.1097/ruq.0000000000000277

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Calabrese A, Santucci D, Landi R, Beomonte Zobel B, Faiella E, de Felice C. Radiomics MRI for lymph node status prediction in breast cancer patients: the state of art. J Cancer Res Clin Oncol (2021) 147(6):1587–97. doi: 10.1007/s00432-021-03606-6

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Zhang J, Li L, Zhe X, Tang M, Zhang X, Lei X, et al. The diagnostic performance of machine learning-based radiomics of DCE-MRI in predicting axillary lymph node metastasis in breast cancer: A meta-analysis. Front Oncol (2022) 12:799209. doi: 10.3389/fonc.2022.799209

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Cronin P, Kelly AM, Altaee D, Foerster B, Petrou M, Dwamena BA. How to perform a systematic review and meta-analysis of diagnostic imaging studies. Acad Radiol (2018) 25(5):573–93. doi: 10.1016/j.acra.2017.12.007

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Li Z, Ye J, Du H, Cao Y, Wang Y, Liu D, et al. Preoperative prediction power of radiomics for breast cancer: A systemic review and meta-analysis. Front Oncol (2022) 12:837257. doi: 10.3389/fonc.2022.837257

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Li L, Wu C, Huang Y, Chen J, Ye D, Su Z. Radiomics for the preoperative evaluation of microvascular invasion in hepatocellular carcinoma: A meta-analysis. Front Oncol (2022) 12:831996. doi: 10.3389/fonc.2022.831996

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Lee J, Kim KW, Choi SH, Huh J, Park SH. Systematic review and meta-analysis of studies evaluating diagnostic test accuracy: A practical review for clinical researchers-part II. statistical methods of meta-analysis. Korean J Radiol (2015) 16(6):1188–96. doi: 10.3348/kjr.2015.16.6.1188

PubMed Abstract | CrossRef Full Text | Google Scholar