Nomograms for Predicting Axillary Lymph Node Status Reconciled With Preoperative Breast Ultrasound Images

Introduction: The axillary lymph node (ALN) status of breast cancer patients is an important prognostic indicator. The use of primary breast mass features for the prediction of ALN status is rare. Two nomograms based on preoperative ultrasound (US) images of breast tumors and ALNs were developed for the prediction of ALN status.

Methods: A total of 743 breast cancer cases collected from 2016 to 2019 at the Second Affiliated Hospital of Harbin Medical University were randomly divided into a training set (n = 523) and a test set (n = 220). A primary tumor feature model (PTFM) and ALN feature model (ALNFM) were separately generated based on tumor features alone, and a combination of features was used for the prediction of ALN status. Logistic regression analysis was used to construct the nomograms. A receiver operating characteristic curve was plotted to obtain the area under the curve (AUC) to evaluate accuracy, and bias-corrected AUC values and calibration curves were obtained by bootstrap resampling for internal and external verification. Decision curve analysis was applied to assess the clinical utility of the models.

Results: The AUCs of the PTFM were 0.69 and 0.67 for the training and test sets, respectively, and the bias-corrected AUCs of the PTFM were 0.67 and 0.67, respectively. Moreover, the AUCs of the ALNFM were 0.86 and 0.84, respectively, and the bias-corrected AUCs were 0.85 and 0.81, respectively. Compared with the PTFM, the ALNFM showed significantly improved prediction accuracy (p < 0.001). Both the calibration and decision curves of the ALNFM nomogram indicated greater accuracy and clinical practicality. When the US tumor size was ≤21.5 mm, the Spe was 0.96 and 0.92 in the training and test sets, respectively. When the US tumor size was greater than 21.5 mm, the Sen was 0.85 in the training set and 0.87 in the test set. Our further research showed that when the US tumor size was larger than 35 mm, the Sen was 0.90 in the training set and 0.93 in the test set.

Conclusion: The ALNFM could effectively predict ALN status based on US images especially for different US tumor size.

Introduction

Axillary lymph node (ALN) status has been considered to be the most important prognostic indicator for breast cancer and the main determinant of adjuvant systemic therapy (1, 2). However, the role of ALN status in the treatment of breast cancer is still a controversial topic. In addition, breast surgery tends to be less invasive and painless (3). ALN dissection (ALND), sentinel lymph node biopsy (SLNB) and axillary sampling are currently used for pathological examination of the ALNs. Although less commonly reported, secondary injury involving anaphylaxis, infection, hematoma, seroma and paresthesia could lead to a prolonged surgical time and financial burden to the patients (4); even axillary biopsy is an invasive procedure. With the increasing detection of small-size or early-stage breast cancer, the incidence of lymph node metastasis has decreased worldwide. Previous studies showed that approximately 65% to 75% of sentinel lymph nodes (SLNs) are negative; thus, SLNB may not be necessary for those patients, which has become a controversial issue (5, 6).

In recent years, medical researchers have attempted to explore some noninvasive methods for predicting SLN status preoperatively to avoid unnecessary ALND, SLNB or lymph node puncture biopsy. The diagnostic value of mammograms and computed tomography (CT) for predicting ALN status is not high (7). Both magnetic resonance imaging (MRI) and positron emission tomography-computed tomography (PET-CT) are not common imaging methods for breast lesions because of their high cost and lack of universal coverage (8, 9). Therefore, ultrasound (US) is widely used in the preoperative examination of breast cancer lesions. With the development of US technology, the strategy of ALN management has been further improved. Preoperatively, determining the ALN status can improve personal and multidisciplinary treatment plans. Currently, there is no unified diagnostic standard for US detection of ALN status. A large number of clinical trials have confirmed that although ALNs cannot be found by US, ALN metastasis may be present (10, 11). In several studies, scholars applied preoperative and postoperative data to establish an SLN metastasis prediction model, and the performance could reach 0.70 to 0.79 (12–14). The prediction effect was not ideal, especially for postoperative prediction, with little clinical application value.

This research aimed to explore the preoperative sonographic features of ALNs and breast masses and clinicopathological features related to ALN status to build two nomograms for predicting ALN status. These models were expected to have better accuracy than the previous methods, provide a more valuable reference for clinical treatment, and make it possible to avoid invasive axillary surgery (ALND, SLNB or lymph node biopsy) to minimize trauma.

Methods

A total of 1059 patients with breast tumors were collected from the Second Affiliated Hospital of Harbin Medical University from Nov. 10, 2016, to Dec. 30, 2019. For breast masses that were suspected to be malignant preoperatively, a preoperative core needle biopsy of the breast masses and suspicious ALNs could be performed first. Neoadjuvant therapy (neoadjuvant chemotherapy, targeted therapy and endocrine therapy) could be used for patients with large masses, those who needed breast conservative surgery, and those with local late-stage disease and stage II disease with poor molecular biological characteristics. For patients with operable masses, a comprehensive surgical treatment (modified radical mastectomy, mastectomy and SLNB, breast conservative surgery, and reconstruction) could be selected based on the patient’s condition. Postoperative radiotherapy, chemotherapy, targeted therapy, endocrine therapy, immunotherapy, and traditional Chinese medicine treatment were selected according to the patient’s condition. The procedure used for patient selection is shown in Figure 1. The inclusion criteria were as follows: invasive breast cancer patients with complete clinical information and US images, patients with preoperative breast mass core needle biopsy, and those with ALNs with SLNB, ALND or core needle biopsy, the patients selected in our study were all before neoadjuvant therapy or not received neoadjuvant therapy. The exclusion criteria were as follows: benign lesions, noninvasive carcinoma, and presence of other malignant tumors. According to the above criteria, 743 patients were finally selected for model construction. US examinations were performed by a board-certified sonographer with 8 years of experience in breast US imaging using a Supersonic Imagine Aixplorer (France) coupled with linear probe arrays of 10 to 2 MHz and 15 to 4 MHz. To maintain consistency, after scanning the entire breast and ipsilateral ALN using B-mode US and color Doppler flow imaging (CDFI), the sonographer selected images of the target tumor and ALN in sagittal and transverse planes and stored dynamic and static images for a double-blind analysis.

FIGURE 1

Figure 1 Recruitment process for study patients.

The expression levels of the estrogen receptor (ER) and progesterone receptor (PR, positive, >1% of cells staining positive) (15), human epidermal growth factor receptor 2 (HER-2/neu, positive, score of 3+ on immunohistochemistry, score of 2+ on fluorescence in situ hybridization) (16), and Ki-67 index (threshold over 14%) (17) were determined by immunohistochemical staining by applying routine methods after preoperative core needle biopsy. Core needle biopsy or mastectomy was used to determine breast pathology. SLNB, ALND, or core needle biopsy was used to determine macrometastasis of the ALNs.

Analysis of US Features

Gray-scale US features were evaluated according to the uniform American College of Radiology Breast Imaging-Reporting and Data System (ACR-BI-RADS) (18), and the color Doppler flow grade was determined according to the Alder classification criteria (19). To ensure consistency of the image analysis results, two sonographers with 10 and 12 years of clinical experience with breast US were responsible for independent retrospective analysis of the images. In cases of disagreement, a consensus was reached. Because there is no unified diagnostic standard for ALN ultrasound, some high-level related literature was consulted (20–22). All features are shown in Table 1. Based on the above features, we developed the following two models: the primary tumor feature model (PTFM) was established using clinical features and US features of primary breast tumors, and the ALN features model (ALNFM) was developed by considering clinical features and US features of primary breast tumors and US features of ALNs. The optimal model was screened out and extensively studied according to tumor size.

TABLE 1

Table 1 Patients: all patients, patients according to training set or test set and comparison of the composition ratio between groups.

Statistical Analysis

A data analysis framework was proposed. Descriptive statistics were applied to assess the frequency distribution among all patients in the study. Interobserver comparisons were evaluated with Cohen’s kappa statistics (23). According to the type of data, the Chi-square test, Mann-Whitney U test and independent t-test were used to compare the two groups (training set vs test set). Univariate logistic regressions were used to evaluate the relationship between each feature and ALN status. Features (P<0.05) in the univariate logistic regression analysis that were assessed with the stepwise logistic regression method were used to construct the nomograms. According to the size of the regression coefficient of all independent variables, the scoring standard was established. The risk of predicting ALN status curves and the features value were plotted according to the scales of these scores. Then, a nomogram was developed by plotting a vertical line from the scale of the total score to the scale of the predicted probability.

Using receiver operating characteristic (ROC) curve analysis, the area under the curve (AUC) was obtained, and the Hosmer-Lemeshow (HL) test was applied to assess the fitting accuracy of the final model. The AUC comparison between the two models was performed by DeLong’s test. In addition, ROC curve analysis was used to select the cut-off point of tumor diameter according to ALN status. The bias-corrected AUC and calibration curves were obtained by the bootstrap resampling method and applied for internal and external verification. Decision curve analysis was used to assess the clinical utility of the prediction nomogram. The odds ratio (OR) and 95% confidence interval were used to quantify the association between the features and lymph node status. The conventional statistical analyses were two-sided. Every feature with a P value < 0.05 was considered statistically significant, and P values were obtained from the Wald test. The statistical software R version 3.5.0, 2.15.3 (http://www.R-project.org/) and Bioconductor packages (http://www.bioconductor.org/) were used for all statistical analyses.

Results

General Results

Mastectomy, breast conservative surgery, reconstruction, and neoadjuvant therapy were performed for 75.23% (559/743), 9.29% (69/743), 0.27% (2/743), and 15.21% (113/743) of patients, respectively. In total, 384 and 246 patients received ALND and SLNB, respectively. Preoperative ALN core needle biopsy was carried out in 438 cases; 113 received neoadjuvant therapy, whereas 325 patients did not receive neoadjuvant therapy but underwent ALND or SLNB. A total of 743 patients were randomly divided into two groups: the training set (n=523) and the test set (n=220). The clinical and US characteristics of the patients are summarized in Table 1. There were no significant differences between the training and test sets (P>0.05). The unweighted values of interobserver agreement determined by Cohen’s kappa statistics ranged from 0.65 to 0.94, which showed that there was good consistency among the observers.

Critical Predictors Obtained by Logistic Regression Analysis

Based on the univariate analysis of 20 variables for the training set (US tumor size, quantity, shape, orientation, margin contour, main posterior echo, microcalcification, echogenicity pattern, CDFI, HER-2 status, US of the lymph node, aspect ratio, cortical thickness, and central hilum), 14 predictive variables were significantly associated with ALN status (P<0.05). Age, margin, boundary, ER status, PR status, and Ki-67 index did not correlate with lymph node status (P≥0.05), as summarized in Table 2.

TABLE 2

Table 2 Critical predictors obtained by univariate logistic regression analysis.

Finally, the results of the PTFM are presented in Table 3. The following features showed a statistically significant relationship with ALN status: US tumor size, shape, orientation, CDFI, and HER-2 status. The P values obtained by applying the HL test were 0.214 and 0.865 in the training and test sets, respectively, which did not indicate a significant difference. The original assumption was accepted, and there was no significant difference between the predicted value and the real value.

TABLE 3

Table 3 Results of the PTFM.

The results of the ALNFM are displayed in Table 4. The following features demonstrated a significant difference that correlated independently with ALN status: US tumor size, orientation, HER-2 status, US of the lymph node, cortical thickness and central hilum. The P value obtained by applying the HL test was 0.369 in the training set and 0.045 in the test, which did not correspond to a significant difference. The results showed that the original assumption was accepted, and there was no difference between the predicted and actual values in the model.

TABLE 4

Table 4 The results of ALNFM.

Nomograms for Predicting ALN Status

The stepwise regression analyses were designed to facilitate the generation of two nomograms for predicting ALN status (Figure 2). Irregular shape (Figure 3A), multiple tumors and higher CDFI classification (Figure 3B) were greatly associated with ALN metastasis in the PTFM nomogram. In both nomograms, nonparallel orientation growth (Figure 3A), main posterior shadowing, larger tumor size, microcalcification (Figure 3C), and HER-2 gene overexpression correlated highly positively with ALN status. In addition, a tumor margin contour that was angular or spiculate, no lymph nodes found, aspect ratio<2, thickened lymph node cortex, and absent central hilum (Figure 3D) correlated highly positively with ALN metastasis in the ALNFM.

FIGURE 2

Figure 2 (A) The PTFM nomogram for predicting ALN status in the training set. (B) The ALNFM nomogram for predicting ALN status in the training set. Sin, Single; Mul, Multiple; O, Oval; R, round; Irr, Irregular; Par, Parallel; NP, Not parallel; Smo, Smooth; Lob, Lobulate; A, Angular; S, Spiculate; E, Enhancement; I, Indifferent; Sha, Shadowing; Abs, Absent; Pre, Preserved; N, Negative; P, Positive; NLNF, No lymph nodes found; LNF, Lymph nodes found; AR, Aspect ratio; T, Thickness; ST, Symmetric thickness; AT, Asymmetric thickness; PA, Partially absent; CA, Completely absent.

FIGURE 3

Figure 3 (A) US image from a 64-year-old woman with ALN metastasis showing the irregular shape and nonparallel growth of the tumor. (B) US images from a 38-year-old woman with ALN metastasis showing rich blood flow signals with the application of color Doppler flow. (C) US images in a 40-year-old woman with ALN metastasis, which shows microcalcification and an irregular shape. (D) US images from a 62-year-old woman with ALN metastasis showing the absence of a central hilum.

Reliability and Validation of the Nomograms

The discriminatory performance of the nomograms for predicting ALN status was evaluated by ROC curves. The sensitivity (Sen), specificity (Spe), and accuracy (Acu) of the PTFM were 0.70, 0.68 and 0.69, respectively, in the training set, and 0.70, 0.64 and 0.67, respectively, in the test set. The AUC values of the PTFM in the training set and the test set were 0.69 (95% CI, 0.65–0.73) and 0.670 (95% CI, 0.61–0.73), respectively (Figure 4A). Moreover, internal and external verification were performed using a bootstrapping method with 1000 resamples in the training and test sets. After bias correction, the AUC of the PTFM was 0.67 (−0.025) and 0.67 (−0.001) in the training set and the test set, respectively. Calibration using a bootstrapping method with 1000 resamples was also estimated graphically in the training set (Figure 5A) and test set (Figure 5B).

FIGURE 4

Figure 4 (A) ROC curves of the PTFM in the training set (green curve) and test set (red curve). (B) ROC curves of the ALNFM in the training set (green curve) and test set (red curve).

FIGURE 5

Figure 5 The calibration curves of the PTFM nomogram in the training set (A) and test set (B). The calibration curves of the ALNFM nomogram in the training set (C) and test set (D). The 45-degree diagonal line demonstrates perfect matching between the nomogram-predicted probabilities (X-axis) and the true value (Y-axis). The distances were closer between the two lines, and the prediction accuracies of the ALNFM nomogram were higher than those of the PTFM.

The Sen, Spe and Acu of the ALNFM were 0.78, 0.93 and 0.86, respectively, in the training set, and 0.80, 0.88 and 0.84, respectively, in the test set. The consistent AUCs of the ALNFM in the training set the test set were 0.86 (95% CI, 0.83–0.89) and 0.84 (95% CI, 0.79–0.89), respectively (Figure 4B). Moreover, internal and external validations were performed using a bootstrapping method with 1000 resamples in the training and test sets. After bias correction, the AUC of the ALNFM was 0.85 (−0.003) and 0.81 (−0.034) in the training and test sets, respectively. Calibration using a bootstrapping method with 1000 resamples was also estimated graphically in the training set (Figure 5C) and test set (Figure 5D). The distance between the two lines of the ALNFM was less than that of the PTFM. Compared with the PTFM, the ALNFM achieved the best performance in both the training and test sets (P<0.001).

Prediction Performance of the ALNFM Using US Tumor Size

ROC curve analysis was used to select the cut-off point of tumor diameter according to ALN status, and the cut-off value was 21.5mm (Supplementary Figure 1). The cut-off values of the Sen and Spe were assessed by application of the ALNFM. When the US tumor size was ≤21.5 mm, the Spe was 0.96 and 0.92 in the training and test sets, respectively. Moreover, with the decrease in US tumor size, the Spe increased significantly. When the US tumor size was greater than 21.5mm, the Sen was 0.85 in the training set and 0.87 in the test set. Furthermore, when the US tumor size increased, the Sen also increased significantly. Our further research showed that when the US tumor size was larger than 35mm, the Sen was 0.90 in the training set and 0.93 in the test set (Supplementary Table 1).

Clinical Value of Evaluation of the Nomograms

Decision curve analysis was used to assess the clinical utility of the two prediction nomograms (Figure 6). It showed that there was an ideal high net benefit. The nomogram constructed with the ALNFM showed the most clinical benefits for predicting ALN status when the risk threshold probability for patients ranged from 0.07 to 0.99. The nomogram constructed with the ALNFM showed more benefit than the nomogram constructed with the PTFM. Therefore, our nomogram constructed with the ALNFM performed better.

FIGURE 6

Figure 6 Decision curve analysis of the two nomograms. The net benefit is shown by the y-axis. The advantages minus disadvantages are equal to the net benefits. When the risk threshold probability is between 0.07 and 0.99, the ALNFM nomogram (red curve) showed more benefit for patients for predicting ALN status than the PTFM nomogram or the treat-all and treat-none schemes.

Discussion

ALN status has been considered an important factor for determining treatment and prognosis of patients (25, 26). Currently, the treatment of breast cancer patients is developing toward to a less invasive treatment of the ALNs, involving less pain and close follow-up (3). If the ALN status can be effectively predicted preoperatively, the surgical plan can be optimized and unnecessary invasive examination of the ALNs can be omitted to reduce trauma. US imaging for diagnosis of ALN status has become a concern in recent years, but this issue is still in the early stages of research, and there is no uniform diagnostic standard. Few studies have been performed on the US features of breast masses to predict ALN status; therefore, we attempted to use easily acquired clinical features and preoperative US features of breast masses and ALNs to predict ALN status and develop forecasting nomograms. A profound study of new methods of traditional US has been performed, which should be brought to the attention of clinical doctors. Further research on traditional US still has a broad application prospect and high application value with the current rapid development of US technology.

To compare the performance of the prediction model of clinical features and US features of primary breast tumors with the model combined with US features of ALNs, we established two nomograms in this study. We found that the ALNFM was more significant than the PTFM (p<0.001). The PTFM nomogram included the characteristics of breast tumors and did not include the characteristics of the ALNs. This nomogram used an indirect method, while the ALNFM used a more direct method. Thus, the PTFM nomogram was less significant than the ALNFM nomogram. However, except for the association of the same features (nonparallel orientation growth, main posterior shadowing, microcalcification, larger tumor size, and HER-2 gene overexpression) with ALN metastasis, the PTFM nomogram incorporated more US features of primary breast tumors, such as multiple tumors, irregular tumors, and abundant blood supply. Sonographers should be reminded to pay special attention to the status of ALNs when they find the above features during a scan.

The performance of the ALNFM (AUC: 0.86 and 0.84 in the training set and the test set, respectively) was not only better than that of the PTFM, but it was also better than that of other previous methods, including dynamic contrast-enhanced MRI, clinicopathological features and the previous Memorial Sloan-Kettering Cancer Center (MSKCC) model, and the AUCs of previous studies ranged from 0.67 to 0.82 (12, 13, 27). The MSKCC model has been verified in other countries in succession, and the AUC ranged from 0.63 to 0.78 (28–30). Meanwhile, our results were also reliable and objective. First, there was little bias in the AUC values in the internal and external verification. The calibration curves were closer to the 45 degree diagonal line, and the prediction accuracies were higher. Second, decision curve analysis showed a higher clinical application value and better clinical practicability. The results of some studies were also consistent with those of our study, with AUCs ranging from 0.85 to 0.86 (31, 32). The nomogram showed that nonparallel orientation growth, a tumor margin contour that was angular or spiculate, main posterior shadowing, larger tumor size, microcalcification, aspect ratio<2, thickened lymph node cortex, absent central hilum, and HER-2 gene overexpression were likely to be associated with ALN metastasis. Therefore, if a breast tumor is detected by preoperative US and accompanied by any of the above conditions, especially larger tumor size, thickened lymph node cortex, or absent central hilum, metastasis of the ALNs could be predicted, and preoperative core needle biopsy, SLNB or ALND should be performed. The Spe of the ALNFM was higher (0.93 and 0.88 in the training and test sets, respectively). Moreover, if the breast tumor was not accompanied by any of the above conditions and the ALNs were predicted to be negative, unnecessary preoperative core needle biopsy, SLNB or ALND may be avoided to a certain extent. Despite the rapid development of US technology, traditional US is still dominant and irreplaceable; at the same time, traditional US combined with clinicopathological characteristics for predicting ALN is currently the most practical and concise new method and is easier to perform and more stable than the radiomics method or elastography (33, 34). In particular, US is a noninvasive and routine method for evaluating breast cancer and can be widely used, even in some underdeveloped areas.

After the best model was selected, we performed in-depth research on the clinical significance of the ALNFM nomogram developed from traditional US and further explored the critical US features associated with ALN metastasis. This study showed that the US tumor size was associated with ALN metastasis, and the results were in agreement with those of a previous study (35, 36). Furthermore, the threshold was 21.5 mm, which was surprisingly similar to the endpoint value (20 mm) of T1 and T2 and this result was consistent with those of previous studies that showed that a tumor size greater than 20 mm was an important independent risk factor for ALN metastasis (37). The study showed that when the US tumor size was ≤21.5 mm, the Spe was ideal(with values of 0.96 and 0.92 in the training and test sets, respectively); when the US tumor size was greater than 21.5 mm, the Sen was very high (0.85 and 0.87 in the training and test sets, respectively); when the US tumor size was larger than 35 mm, the Sen was ideal(with values of 0.90 and 0.93 in the training and test sets, respectively), and previous studies believe that it was ideal when the false negative rate (1-sensitivity) of a model was less than 10% (38). Thus, when the US tumor size was ≤21.5 mm, the ALN status was predicted to be negative, SLNB or core needle biopsy may not be recommend for surgical staging, and close follow-up should be recommend to some small tumors. when the US tumor size was >21.5 and ≤35 mm, the ALN procedure was selected based on the combination of clinical and ultrasound features related to ALN metastasis. If the US tumor size was greater than 35mm and ALN is predicted to be negative, invasive ALN biopsy, sentinel lymph node biopsy and dissection may be avoided and when the ALN status was predicted to be positive, core needle biopsy, SLNB or ALND was absolutely necessary. Thus, the clinical application value of the ALNFM according to US tumor size was obviously significant.

Our study indicated that the shape, orientation and CDFI were markedly independently associated with ALN metastasis. The presence of multiple tumors, a margin contour that was angular or spiculate, posterior echo shadowing, hypoechoic and microcalcification were obviously associated with ALN metastasis based on univariate logistic regressions analysis. The results were consistent with those of previous studies (22, 27). We thought that the results might be related to the biological behavior of tumors; the more likely a tumor is to metastasize, the faster it grows, the larger it grows, the greater the number of tumors, the more unbalanced it is, the more angular or spiculate it is, the more irregular its shape is, the more likely it is to grow in a nonparallel orientation, and the more abundant the blood supply is. Furthermore, because it is growing too fast, the nutrition is often in poor supply, so the tumor is more prone to microcalcification and more likely to be accompanied by posterior echo shadowing (39). The study also showed that aspect ratio<2, thickened lymph node cortex and absent central hilum were independently positively associated with ALN metastasis. Clinical studies have confirmed that cancer cells transfer to the lymph nodes and destroy the cortex, medulla and central hilum, leading to enlarged ALNs, a thickened cortex, enhanced lateral growth, and a diminished aspect ratio (40, 41). When the ALNs were found by ultrasound, the impacts of ALN features on the predicting results will be highlighted. Our research found that even if ALNs were not found by ultrasound, there were 22.90% (49/214) ALN metastases. When the ALNs were not found by ultrasound, the impacts of the ALN features on the predicting results were absent, therefore, the impacts of No lymph nodes found on the predicting results will be highlighted. These may explain why no lymph nodes found correlated highly positively with ALN metastasis. This study also confirmed that the molecular phenotype was related to ALN status, and patients with HER-2 positivity had a higher metastasis risk, which was consistent with the results of other experts who found that HER-2 gene overexpression in breast cancer was associated with a higher degree of malignancy and an increased likelihood of metastasis (24).

Regardless of the development of US technology in the future, the simple and easy-to-use traditional US always has broad application prospects in predicting ALN status. Compared with difficult, challenging, and expensive methods, US has great importance in clinical practice; moreover, it had greater guiding significance in the procedure of ALNs than previous studies and postoperative methods. Therefore, irrespective of how many models there are, application value is the most important determinant.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by the Ethics Committee of the Second Affiliated Hospital of Harbin Medical University. The patients/participants provided their written informed consent to participate in this study. Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.

Author Contributions

DL, JT, and XL designed and guided the study. DL and LZ analyzed the imaging data. TW collected the ultrasound images. HC, ZL, PS, and PT collected the clinical data. DL, YL, and XL performed the statistical analysis. DL wrote the manuscript. JT revised the manuscript. All authors contributed to the article and approved the submitted version.

Funding

This work was supported by the National Natural Science Foundation of China under grants 81630048, 81701705, 81974265, and 81901761; postdoctoral Funding of Heilongjiang Province under grant LBH-Z17174; and scientific research project of Heilongjiang Health Committee no. 2019-050.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fonc.2021.567648/full#supplementary-material

Supplementary Figure 1 | ROC curve of tumor diameter with ALN status.

References

1. Haagensen CD. Treatment of curable carcinoma of the breast. Int J Radiat Oncol Biol Phys (1977) 2(9-10):975–80. doi: 10.1016/0360-3016(77)90198-5

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Pinuccia Valacussa BGBM. Patterns of Relapse and Survival Following Radical Mastectomy. Cancer (1978) 41(3):1170–8. doi: 10.1002/1097-0142. http://www.Bonadonnahttp://www.U Veronesi.

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Ollila DW, Hwang ES, Brenin DR, Kuerer HM, Yao K, Feldman S, et al. The Changing Paradigms for Breast Cancer Surgery: Performing Fewer and Less-Invasive Operations. Ann Surg Oncol (2018) 25(10):2807–12. doi: 10.1245/s10434-018-6618-z

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Leiter U, Stadler R, Mauch C, Hohenberger W, Brockmeyer N, Berking C, et al. Complete lymph node dissection versus no dissection in patients with sentinel lymph node biopsy positive melanoma(DeCOG-SLT):a multicentre, randomised, phase 3 trial. Lancet Oncol (2016) 17(6):757–67. doi: 10.1016/S1470-2045(16)00141-8

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Krag DN, Anderson SJ, Julian TB, Brown AM, Harlow SP, Ashikaga T, et al. Technical outcomes of sentinel-lymph-node resection and conventional axillary-lymph-node dissection in patients with clinically node-negative breast cancer:results from the NSABP B-32 randomised phase III trial. Lancet Oncol (2007) 8(10):881–8. doi: 10.1016/S1470-2045(07)70278-4

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Giuliano AE, Jones RC, Brennan M, Statman R, Giuliano AE. Sentinel Lymphadenectomy in Breast Cancer. J Clin Oncol (1997) 15(6):2345–50. doi: 10.1200/JCO.1997.15.6.2345

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Uematsu T, Sano M, Homma KX. In vitro high-resolution helical CT of small axillary lymph nodes in patients with breast cancer: correlation of CT and histology. AJR Am J Roentgenol (2001) 176(4):1069–74. doi: 10.2214/ajr.176.4.1761069

PubMed Abstract | CrossRef Full Text | Google Scholar

8. García Vicente AM, Soriano Castrejón Á, León Martín A, Relea Calatayud F, Muñoz Sánchez MD, Cruz Mora MÁ, et al. Early and delayed prediction of axillary lymph node neoadjuvant response by (18)F-FDG PET/CT in patients with locally advanced breast cancer. Eur J Nucl Med Mol Imaging (2014) 41(7):1309–18. doi: 10.1007/s00259-013-2657-7

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Memarsadeghi M, Riedl CC, Kaneider A, Galid A, Rudas M, Matzek W, et al. Axillary lymph node metastases in patients with breast carcinomas: assessment with nonenhanced versus uspio-enhanced MR imaging. Radiology (2006) 241(2):367–77. doi: 10.1148/radiol.2412050693

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Alvarez S, Añorbe E, Alcorta P, Alcorta P, López F, Alonso L, et al. Role of sonography in the diagnosis of axillary lymph node metastases in breast cancer: a systematic review. AJR Am J Roentgenol (2006) 186(5):1342–8. doi: 10.2214/AJR.05.0936

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Strauss HG, Lampe D, Methfessel G, Buchmann J. Preoperative axilla sonography in breast tumor suspected of malignancy–a diagnostic advantage? Ultraschall Med (1998) 19(2):70–7. doi: 10.1055/s-2007-1000464

PubMed Abstract | CrossRef Full Text | Google Scholar

12. José Luiz B, Michael W, Jane V, Hiram S, Patrick I, Kimberly J. Doctor, What Are My Chances of Having a Positive Sentinel Node A Validated Nomogram for Risk Estimation. J Clin Oncol (2007) 25:3670–9. doi: 10.1200/JCO.2006.08.8013.37%

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Dihge L, Bendahl P-O, Rydén L. Nomograms for preoperative prediction of axillary nodal status in breast cancer. Br J Surg (2017) 104(11):1494–505. doi: 10.1002/bjs.10583.35.8

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Houvenaeghel G, Lambaudie E, Classe JM, Mazouni C, Giard C, Cohen M, et al. Lymph node positivity in different early breast carcinoma phenotypes: a predictive model. BMC Cancer (2019) 19(1):45. doi: 10.1186/s12885-018-5227-3.34%

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Hammond ME, Hayes DF, Dowsett M, Allred CD, Hagerty KL, Badve S, et al. American Society of Clinical Oncology/College of American Pathologists guideline recommendations for immunohistochemical testing of estrogen and progesterone receptors in breast cancer. J Clin Oncol (2010) 28(16):2784–95. doi: 10.1200/JOP.777003

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Perez EA, Dueck AC, Mccullough AE, Reinholz MM, Tenner KS, Davidson NE, et al. Predictability of adjuvant trastuzumab benefit in N9831 patients using the ASCO/CAP HER2-positivity criteria. J Natl Cancer Inst (2012) 104(2):159–62. doi: 10.1093/jnci/djr490

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Cheang M C, Chia SK, Voduc D, Gao D, Leung S, Snider J, et al. Ki67 index, HER2 status, and prognosis of patients with luminal B breast cancer. J Natl Cancer Inst (2009) 101(10):736–50. doi: 10.1093/jnci/djp082

PubMed Abstract | CrossRef Full Text | Google Scholar

18. D’Orsi CJ, Sickles EA, Mendelson EB, Morris EA. ACR BI-RADS Atlas: Breast Imaging Reporting and Data System. 5th ed. Reston: VA: American College of Radiology (2013).

Google Scholar

19. Adler DD, Carson PL, Rubin JM, Reid DQ. Doppler ultrasound color flow imaging in the study of breast cancer: preliminary findings. Ultrasound Med Biol (1990) 16(6):553–9. doi: 10.1016/0301-5629(90)90020-d

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Koelliker SL, Chung MA, Mainiero MB, Steinhoff MM, Cady B. Axillary lymph nodes: US-guided fine-needle aspiration for initial staging of breast cancer-Correlation with primary tumor size. Radiology (2008) 246(1):81–9. doi: 10.1148/radiol.2463061463

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Gruber I, Hahn M, Fehm T, Weyhern C, Stäbler A, Winckelmann A, et al. Relevance and methods of interventional breast sonography in preoperative axillary lymph node staging. Ultraschall Med (2012) 33(4):337–43. doi: 10.1055/s-0031-1273317

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Koehler KE, Ohlinger R. Sensitivity and specificity of preoperative ultrasonography for diagnosing nodal metastases in patients with breast cancer. Ultraschall Med (2011) 32(4):393–9. doi: 10.1055/s-0029-1245505.32%

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Svanholm H, Starklint H, Gundersen HJ, Fabricius J, Barlebo J, Olsen S. Reproducibility of histomorphologic diagnoses with special reference to the kappa statistic. APMIS (1989) 97(8):689–98. doi: 10.1111/j.1699-0463.1989.tb00464.x

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Demonty G, Bernard-Marty C, Puglisi F, Mancini S, Piccart M. Progress and new standards of care in the management of HER-2 positive breast cancer. Eur J Cancer (2007) 43(3):497–509. doi: 10.1016/j.ejca.2006.10.020

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Eriksson L, Bergh J, Humphreys K, Wärnberg F, Törnberg S, Czene K, et al. Time from breast cancer diagnosis to therapeutic surgery and breast cancer prognosis: A population-based cohort study. Int J Cancer (2018) 143(5):1093–104. doi: 10.1002/ijc.31411

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Colzani E, Liljegren A, Johansson AL, Adolfsson J, Hellborg H, Hall PF, et al. Prognosis of patients with breast cancer: causes of death and effects of time since diagnosis, age, and tumor characteristics. J Clin Oncol (2011) 29(30):4014–21. doi: 10.1200/JCO.2010.32.6462

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Liu C, Zhao Z, Gu X, Sun L, Chen G, Zhang H, et al. Establishment and Verification of a Bagged-Trees-Based Model for Prediction of Sentinel Lymph Node Metastasis for Early Breast Cancer Patients. Front Oncol (2019) 9:282. doi: 10.3389/fonc.2019.00282

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Van den Hoven I, Kuijt GP, Voogd AC, Beek MW, Roumen RM. Value of Memorial Sloan-Kettering Cancer Center nomogram in clinical decision making for sentinel lymph node-positive breast cancer. Br J Surg (2010) 97(11):1653–8. doi: 10.1002/bjs.7186

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Van La Parra RF, Francissen CM, Peer PG, Ernst MF, Roos WK, Van Zee KJ, et al. Assessment of the Memorial Sloan-Kettering Cancer Center nomogram to predict sentinel lymph node metastases in a Dutch breast cancer population. Eur J Cancer (2013) 49(3):564–71. doi: 10.1016/j.ejca.2012.04.025

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Rouzier R, Uzan C, Rousseau A, Guillot E, Zilberman S, Meyer C, et al. Multicenter prospective evaluation of the reliability of the combined use of two models to predict non-sentinel lymph node status in breast cancer patients with metastatic sentinel lymph nodes: the MSKCC nomogram and the Tenon score. Results of the NOTEGS study. Br J Cancer (2017) 116(9):1135–40. doi: 10.1038/bjc.2017.47

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Qiu SQ, Zeng HC, Zhang F, Chen C, Huang WH, Pleijhuis RG, et al. A nomogram to predict the probability of axillary lymph node metastasis in early breast cancer patients with positive axillary ultrasound. Sci Rep (2016) 15(6):21196. doi: 10.1038/srep21196

CrossRef Full Text | Google Scholar

32. de Camargo Teixeira PA, Chala LF, Shimizu C, Filassi JR, Maesaka JY, Barros N. Axillary Lymph Node Sonographic Features and Breast Tumor Characteristics as Predictors of Malignancy: A Nomogram to Predict Risk. Ultrasound Med Biol (2017) 43(9):1837–45. doi: 10.1016/j.ultrasmedbio.2017.05.003

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Sun Q, Lin X, Zhao Y, Li L, Yan K, Liang D, et al. Deep Learning vs. Radiomics for Predicting Axillary Lymph Node Metastasis of Breast Cancer Using Ultrasound Images: Don’t Forget the Peritumoral Region. Front Oncol (2020) 10:53. doi: 10.3389/fonc.2020.00053

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Li X-L, Xu H-X, Li D-D, He YP, Yue W-W, Xu J-M, et al. A Risk Model Based on Ultrasound,Ultrasound Elastography,and Histologic Parameters for Predicting Axillary Lymph Node Metastasis in Breast Invasive Ductal Carcinoma. Sci Rep (2017) 7(1):3029. doi: 10.1038/s41598-017-03582-3.39.5%

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Hernandez-Aya LF, Chavez-Macgregor M, Lei X, Meric-Bernstam F, Buchholz TA, Hsu L, et al. Nodal status and clinical outcomes in a large cohort of patients with triple-negative breast cancer. J Clin Oncol (2011) 29(19):2628–34. doi: 10.1200/JCO.2010.32.1877

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Chen M, Palleschi S, Khoynezhad A, Gecelter G, Marini CP, Simms HH. Role of primary breast cancer characteristics in predicting positive sentinel lymph node biopsy results: a multivariate analysis. Arch Surg (2002) 137(5):606–9. doi: 10.1001/archsurg.137.5.606

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Viale G, Zurrida S, Maiorano E, Mazzarol G, Pruneri G, Paganelli G, et al. Predicting the Status of Axillary Sentinel Lymph Nodes in 4351 Patients with Invasive Breast Carcinoma Treated in a Single Institution. Cancer (2005) 103(3):492–500. doi: 10.1002/cncr.20809

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Krag DN, Anderson SJ, Julian TB, Brown AM, Harlow SP, Ashikaga T, et al. National Surgical Adjuvant Breast and Bowel Project. Technical outcomes of sentinel-lymph-node resection and conventional axillary-lymph-node dissection in patients with clinically node-negative breast cancer: results from the NSABP B-32 randomised phase III trial. Lancet Oncol (2007) 8(10):881–8. doi: 10.1016/S1470-2045(07)70278-4

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Colpaert C, Vermeulen P, Jeuris W, Beest P, Goovaerts G, Weyler J, et al. Early Distant Relapse in “Node-Negative” Breast Cancer Patients Is Not Predicted by Occult Axillary Lymph Node Metastases, but by the Features of the Primary Tumour. J Pathol (2001) 193(4):442–9. doi: 10.1002/path.829

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Feu J, Tresserra F, Fábregas R, Navarro B, Grases PJ, Suris JC, et al. Metastatic breast carcinoma in axillary lymph nodes in vitro US detection. Radiology (1997) 205(17):1831–5. doi: 10.1148/radiology.205.3.9393544

CrossRef Full Text | Google Scholar

41. Yang WT, Chang J, Metreweli C. Patients with breast cancer: differences in color Doppler flow and gray-scale US features of benign and malignant axillary lymph nodes. Radiology (2000) 215:568–73. doi: 10.1148/radiology.215.2.r00ap20568

PubMed Abstract | CrossRef Full Text | Google Scholar