Novel Nomograms Individually Predicting Overall Survival of Non-metastatic Colon Cancer Patients

Background: This study aimed to develop an effective prognostic nomogram for predicting non-metastatic colon cancer.

Methods: The Surveillance, Epidemiology, and End Results program was utilized to analyze patients who underwent surgical therapy (25,350 for training, 10,860 for validation). Nomograms were created depending upon multivariate analysis in the training cohort and were compared to current American Joint Committee on Cancer (AJCC) classifications. Areas under the receiver-operating characteristic curves (AUCs), Akaike's information criterions (AICs), and calibration curves were used. The clinical benefit was measured using decision curve analyses (DCAs). The validation cohort was used to validate the results.

Results: Nomogram 1 included age, gender, histological grade, T stage, number of retrieved lymph nodes, tumor size, and N stage. Nomogram 2 included age, gender, histological grade, T stage, number of retrieved lymph nodes, tumor size, and number of positive lymph nodes. The prognostic discrimination of nomogram 1 (AUC, 0.729, 95% CI, 0.723–0.736) was better than that of nomogram 2 (AUC, 0.704, 95% CI, 0.698–0.710, p < 0.001) in five-year overall survival in the training cohort. Nomogram 1 (AIC, 137,319) also showed superior model-fitting compared to nomogram 2 (AIC, 137,453). Similarity, nomogram 1 was better than the AJCC 6th and 8th TNM classifications. DCA revealed that nomogram 1 had a superior net benefit than other models. These findings were validated using the validation cohort.

Conclusions: The proposed nomogram 1 was a better prognostic prediction model with better discrimination and superior model-fitting for patients with non-metastatic colon cancer, which might prove to be clinically helpful.

Introduction

Colon cancer is the third most commonly diagnosed cancer among both males and females in the United States (1). Although some progress has been made in the therapy of colon cancer in the past decades (2, 3), local recurrence and distant metastasis remain a challenge for clinicians (4). The accuracy of survival prediction for patients is critical for postoperative treatment decisions and surveillance. Therefore, tools necessary to provide prognosis for colon cancer patients are critical for helping medical professionals consult and advise patients on their treatment options.

The American Joint Committee on Cancer (AJCC) tumor-node-metastasis (TNM) staging system is the current gold standard for risk assessment (5). It is the most basic and common staging system for evaluating prognosis for colon cancer patients undergoing surgery. For colon cancer patients who are not distantly metastatic, the TNM staging system is determined by two factors: the degree of entry into the intestinal wall and the number of locoregional positive lymph nodes (6). In fact, because of the clinicopathological features and tumor biology variations, the outcomes are quite different, so it is assumed that patients in each group have homogenous results (7). In addition, the classification nature of the TNM staging scheme forces continuous variables into categorical variables, might further limiting prediction accuracy (8). It is increasingly recognized that in addition to the TNM staging system, other clinical factors may contribute significantly to individual predictions of prognosis, such as age, histological type, degree of differentiation, systemic inflammation, and nutritional status (9, 10).

A nomogram is an effective tool for visualizing regression models used to quantify individual risk by including multiple important prognostic factors. It has been shown to achieve a good predictive performance in a variety of cancers. Previous studies in pancreatic cancer (11), uveal melanoma (12), hepatocellular carcinoma (13), and intrahepatic cholangiocarcinoma (14) have shown that nomograms could improve predictive accuracy and provide patients and physicians with a more comprehensive outcome measure when making treatment-related decisions.

In previous studies focusing on colorectal cancer, nomograms were applied to predict overall survival, disease-related survival prognosis, risk of recurrence and metastasis, as well as adjuvant chemotherapy. In the development of the nomograms, some studies used number of positive lymph nodes as a continuous variable (8, 15–18), while others used it as a categorical variable (19–23). However, few studies have applied both the AJCC N stage and number of positive lymph nodes (continuous and categorical variables) to develop nomograms, and to compare their discriminations, model-fittings, and net benefits in predicting overall prognosis.

This study aimed to create a prognostic model of colon cancer depending upon independently prognostic factors of Cox proportional-hazards models. Predictive utility of the nomogram was further compared to the AJCC 6th and 8th TNM staging systems (24). This nomogram is expected to provide more personalized prognostic predictions that will help clinicians and patients make better treatment choices.

Materials and Methods

Patients

Using the Surveillance, Epidemiology, and End Results (SEER) program, 691,749 colon cancer patients between 1973 and 2015 were screened (25). The inclusion criteria were as follows: (1) colon cancers from SEER; (2) necessary information available; (3) aged between 18 and 72 years; (4) primary and single tumors; (5) no distant metastasis (M0); (6) received surgery; (7) no preoperative therapy; (8) survival longer than one month; and (9) follow-up ≥ 60 months or until death. Exclusion criteria were as follows: (1) no available information; (2) aged <18 or > 72 years; (3) multiple colon cancers; (4) distant metastasis (M1); (5) no surgical treatment; (6) received preoperative therapy; (7) postoperative survival less than one month; and (8) lost to follow-up or follow-up <60 months. Finally, a total of 36,210 patients were included and randomized into training (70%, n = 25,350) and validation cohorts (30%, n = 10,860).

Statistical Analyses

Baseline clinical variable characteristics between training and validation cohorts were compared with Student's t-tests or Mann-Whitney U tests. Survival curves were depicted using the Kaplan-Meier methods with log-rank tests. Nomograms were developed depending upon prognostic factors of multivariate Cox proportional hazards models.

The predictive discriminations of nomograms and current AJCC TNM classifications were assessed using areas under the receiver-operating characteristic curves (AUCs). The Hanley and McNeil tests were then used to compare the AUCs. The Akaike's information criterion (AIC) (26) and calibration curve (27) were applied to evaluate the nomogram prediction model-fitting. Higher AUCs indicated better discrimination and lower AICs indicated superior model-fitting. The calibration curves were assessed by reviewing the predicted versus actual probabilities. A perfectly accurate classification would result in a calibration curve where most observed and predicted probabilities fall along the 45-degree line. In addition, clinical benefit was measured using decision curve analyses (DCAs) (28, 29).

All data were analyzed using the SPSS 22.0 statistical package (SPSS Inc., Chicago, IL, USA), MedCalc (Version 15.2, Ostend, Belgium), and R version 3.5.6 (http://www.r-project.org/). All tests were two-sided and p-values <0.05 were considered statistically significant. The authors signed a data use agreement with SEER. The approval of an institutional review board was not required as the SEER database holds publicly available deidentified data.

Results

Clinicopathologic Characteristics

Numbers of patients excluded at each step during patient selection process is shown in Figure 1. Patients were categorized according to the AJCC 6th and 8th TNM staging systems. Demographic and clinical characteristics of the training and validation cohorts are shown in Table 1. Baseline characteristics of the validation cohort were similar to the training cohort (Student' t-test or Mann-Whitney U test, p > 0.05 for all).

FIGURE 1

Figure 1. Flow chart for patient selection and study development.

TABLE 1

Table 1. Baseline characteristics of training and validation cohorts.

In the training cohort, the median age was 60 years (range: 18 to 72 years), median tumor size was 42 mm (range: 1 to 150 mm), and median number of retrieved lymph nodes was 16 (range: 1 to 89). A total of 13,275 (52.4%) patients were male and 12,075 (47.6%) were female. According to the AJCC 6th TNM staging system, 5,476 (21.6%) patients were stage I, 7,971 (31.4%) were stage IIA, 1,251 (4.9%) were stage IIB, 1,134 (4.5%) were stage IIIA, 5,577 (22.0%) were stage IIIB, and 3,941 (15.5%) were stage IIIC. According to the AJCC 8th TNM staging system, 5,493 (21.7%) patients were stage I, 8,011 (31.6%) were stage IIA, 667 (2.6%) were stage IIB, 591 (2.3%) were stage IIC, 1,150 (4.5%) were stage IIIA, 6,949 (27.4%) were stage IIIB, and 2,489 (9.8%) were stage IIIC. Univariate analysis identified age, gender, histological grade, AJCC 8th T stage, number of retrieved lymph nodes, tumor size, AJCC 8th N stage, and number of positive lymph nodes significantly associated with overall survival (Table 2). The results were similar in the validation cohort (Supplementary Table S1).

TABLE 2

Table 2. Univariate analysis of the training cohort.

Nomogram 1

Depending upon univariate analysis, statistically significant factors including age, gender, histological grade, T stage, number of retrieved lymph nodes, and tumor size were identified as independent prognostic factors and were included in multivariate analysis of Cox proportional hazards models together with the N stage. Significant factors in the multivariate analysis were further incorporated into nomogram 1 (Table 3, Figure 2A). The results were similar in the validation cohort (Supplementary Table S2).

TABLE 3

Table 3. Multivariable analyses of the training cohort.

FIGURE 2

Figure 2. Nomograms predicting three- and five-year overall survival (OS). For each patient, corresponding clinicopathological feature points were calculated and summed up to obtain total points. Predicted three- and five-year OS can be estimated based on total points for each patient. (A) Nomogram 1: variables included age, gender, histological grade, AJCC 8th T stage, tumor size, retrieved lymph nodes, and AJCC 8th N stage; (B) Nomogram 2: variables included age, gender[[Inline Image]], histological grade, AJCC 8th T stage, tumor size, RLNs, and number of positive lymph nodes.

Nomogram 2

According to univariate analysis, statistically significant factors including age, gender, histological grade, T stage, number of retrieved lymph nodes, and tumor size were included in multivariate analysis of Cox proportional hazards models together with the number of positive lymph nodes. Significant factors in the multivariate analysis were further incorporated into nomogram 2 (Table 3, Figure 2B). The results were similar in the validation cohort (Supplementary Table S2).

Three- and Five-Year Overall Survival of AJCC 6th and 8th Staging Systems

In the training cohort analyzed with the AJCC 6th TNM staging system, the three- and five-year Overall Survival (OS) of stages IIA, IIB, and IIIA was 91.1 and 85.3%, 79.4 and 70.2%, and 92.5 and 87.8%, respectively. The prognosis of stage IIIA was better than that of stages IIA and IIB (Log-rank test, p < 0.05 for all, Figure 3A). The results were similar in the validation cohort (Figure 3C).

FIGURE 3

Figure 3. Kaplan–Meier survival curve based on the AJCC 6th TNM and AJCC 8th TNM classifications. (A) Kaplan–Meier survival curves based on AJCC 6th TNM classification in the training cohort. (B) Kaplan–Meier survival curves based on AJCC 8th TNM classification in the training cohort. (C) Kaplan–Meier survival curves based on AJCC 6th TNM classification in the validation cohort. (D) Kaplan–Meier survival curves based on AJCC 8th TNM classification in the validation cohort.

In the 8th TNM staging system, the three- and five-year OS of stages IIA, IIB, IIC, and IIIA was 91.1 and 85.3%, 81.7 and 72.6%, 76.7 and 67.3%, and 92.4 and 87.4%, respectively. The prognosis of Stage IIIA was better than that of stages IIA, IIB, and IIC (Log-rank test, p < 0.05 for all, Figure 3B). The results were similar in the validation cohort (Figure 3D).

Comparison of Predictive Performance Between two Nomograms

In the training cohort, AUCs of nomogram 1 at three- and five-year OS were 0.740 (95% CI, 0.734–0.746) and 0.729 (95% CI, 0.723–0.736), respectively, with AIC for OS of 137,319 (Table 4, Figures 4A,B). AUCs of nomogram 2 at three- and five-year OS were 0.717 (95% CI, 0.711–0.723) and 0.704 (95% CI, 0.698–0.710), respectively, with AIC for OS of 137,453 (Table 4, Figures 4C,D).

TABLE 4

Table 4. Comparisons of different predictive models of nomograms with AJCC 6th and 8th TNM classifications.

FIGURE 4

Figure 4. Time-dependent areas under receiver-operating characteristic (ROC) curves (AUCs) in training and validation cohorts for three- and five-year OS. In training cohort, (A, B) were nomogram 1 for three- and five-year OS, respectively. (C, D) were nomogram 2 for three- and five-year OS, respectively. (E, F) were AJCC 8th TNM classification for three- and five-year OS, respectively. (G, H) were AJCC 6th TNM classification for three- and five-year OS, respectively. In validation cohort, (I, J) were nomogram 1 for three- and five-year OS, respectively. (K, L) were nomogram 2 in three- and five-year OS, respectively. (M, N) were AJCC 8th TNM classification for three- and five-year OS, respectively. (O, P) were AJCC 6th TNM classification for three- and five-year OS, respectively.

In the validation cohort, AUCs of nomogram 1 at three- and five-year OS were 0.745 (95% CI, 0.739–0.751) and 0.732 (95% CI, 0.729–0.738), respectively, with the AIC for OS of 54,121 (Table 4, Figures 4I,J). AUCs of nomogram 2 at three- and five-year OS were 0.722 (95% CI, 0.716–0.728) and 0.704 (95% CI, 0.698–0.710), respectively, with the AIC for OS of 54,170 (Table 4, Figures 4K,L).

In the training cohort, the prognostic discrimination of nomogram 1 was better than of nomogram 2 (Hanley and McNeil test, all p < 0.001, Table 4) in three- and five-year OS. Nomogram 1 also showed a superior model-fitting compared to nomogram 2 according to the AICs and calibration curves (Table 4, Figures 5A–D). Similar findings were validated in the validation cohort.

FIGURE 5

Figure 5. Calibration curve for predicting patient survival at (A) three and (B) five years in the training cohort and at (C) three and (D) five years in the validation cohort. Nomogram-predicted probability of OS is plotted on the x-axis; actual OS is plotted on the y-axis. Shorter distance between two curves indicates higher accuracy.

Comparison of Predictive Performance Between AJCC 6th and 8th TNM Staging Systems

In the training cohort, the AUCs for the AJCC 8th TNM staging system for three- and five-year OS were 0.703 (95% CI, 0.697–0.709) and 0.695 (95% CI, 0.689–0.701), respectively, with the AIC for OS of 138,404 (Table 4, Figures 4E,F). The AUCs for the AJCC 6th TNM staging system at three- and five-year OS were 0.696, (95% CI, 0.690–0.702) and 0.689 (95% CI, 0.683–0.695), respectively, with the AIC for OS of 138,717 (Table 4, Figures 4G,H).

In the validation cohort, the AUCs for the AJCC 8th classification for three- and five-year OS were 0.698 (95% CI, 0.692–0.704) and 0.690 (95% CI, 0.684–0.696), respectively, with the AIC for OS of 54,628 (Table 4, Figures 4M,N). The AUCs for the AJCC 6th classification at three- and five-year OS were 0.687, (95% CI, 0.681–0.693) and 0.686, (95% CI, 0.680–0.692), respectively, with the AIC for OS of 54,731(Table 4, Figures 4O,P).

In the training cohort, the prognostic discrimination of the AJCC 8th classification was better than of the AJCC 6th classification (Hanley and McNeil test, all p < 0.001, Table 4) in three- and five-year OS. The AJCC 8th classification also showed superior model-fitting compared to the AJCC 6th classification according to the AICs (Table 4). Similar findings were validated in the validation cohort.

Comparison of Predictive Performance Between Nomogram 1 and AJCC 8th TNM Staging Systems

In the training cohort, the prognostic discrimination of nomogram 1 was better than the AJCC 8th classification (Hanley and McNeil test, all p < 0.001, Table 4) in three- and five-year OS in the two cohorts. Nomogram 1 also showed superior model-fitting compared to the AJCC 8th classification according to the AICs in the two cohorts (Table 4). Similar findings were validated in the validation cohort.

Comparison of Clinical Usefulness Between Nomograms and AJCC TNM Staging System Using Decision Curve Analyses

Using the decision curve analyses (DCAs) for both training and validation cohorts, nomogram 1 showed better net benefits with wider ranges of threshold probabilities and improved performance than nomogram 2 and AJCC 6th and 8th TNM staging systems for predicting three- and five-year OS in colon cancer patients (Figures 6A–D). These results represent a superior estimation of decision outcomes at higher threshold probability levels.

FIGURE 6

Figure 6. Decision curve analysis of training and validation cohorts for three- and five-year OS. Decision curve analysis was used to compare clinical net benefits between nomograms and conventional staging systems in terms of three-year OS for (A) training and (C) validation cohorts and five-year OS for (B) training and (D) validation cohorts. For decision curve analysis, horizontal solid black line assumed no patients would die and dotted gray line assumed all patients would die.

Discussion

Accurate predictions of the prognosis for colon cancer patients are critical for further postoperative treatment and follow-up planning. Traditionally, the survival outcome of postoperative colon cancer patients is predicted based on the AJCC TNM staging system. Since the 1940s, the TNM staging system has determined the extent of cancer based only on the depth of tumor infiltration and the number of positive lymph nodes. Revisions of the staging system were modified every six to eight years and, until recently, it has been considered the most comprehensive tool for predicting the prognosis and predictive grouping of colon cancer patients. However, when the AJCC 6th TNM staging system was released in 2002 (24), its accuracy was questioned because the survival rate of patients with stage IIIA was superior to that of patients with stage IIB colon cancer (30). The AJCC 7th edition released in 2010 (31) has been staging more accurately than the AJCC 6th edition for improving the prognosis. However, the AJCC 7th edition has not eliminated survival discrepancies between stages II and IIIA colon cancers. The AJCC 8th edition (6) released in 2017 showed no changes in stages I–III compared to the AJCC 7th TNM staging system. A similar issue was observed in the current study, where the AJCC 6th staging system did not satisfactorily stratify patients between stages II and III. Patient prognosis with stage IIIA was better than that with stage II. The 8th TNM staging system made the staging more elaborate compared to the 6th TNM classification. However, it still does not do a good job at stratifying patients between stages II and III.

In this study, prognostic nomograms based on the results of the Cox proportional-hazards model were developed and validated to predict survival probabilities in patients undergoing surgery for non-metastatic colon cancer. Compared to the 6th and 8th editions of the AJCC staging system based on the depth of infiltration and the number of positive lymph nodes, a nomogram can integrate various prognostic factors to make more personalized predictions for patients. Age, gender, histological grade, T stage, number of retrieved lymph nodes, tumor size, N stage, and number of positive lymph nodes were integrated into the nomogram. Many researchers have also shown that these clinicopathological factors are associated with the prognosis of colon cancer patients (32). It should also be noted that the number of retrieved lymph nodes was an independent factor in the prognosis of colon cancer. Many previous studies have shown that it is also an independent prognostic factor for many other malignancies and the larger number of lymph nodes removed meant a better survival prognosis (33, 34). Perhaps the most important reason is that as the lymph nodes are more extensively removed, more potentially positive lymph nodes will not be missed, providing enough positive lymph nodes to be used for precise staging.

In the establishment of the nomogram, some researchers have used the number of positive lymph nodes as a continuous variable (8, 15–18), while others used it as a categorical variable (19–23). Few researchers have used both the AJCC N stage and number of positive lymph nodes as variables to develop nomograms and to compare their accuracy in predicting prognosis in their studies. The present study developed a nomogram using these variables and evaluated the accuracy in predicting prognosis. The results showed that the nomogram including the AJCC 8th N stage had a better survival prediction accuracy than the nomogram including the number of positive lymph nodes. The nomogram incorporates clinically common pathological factors and provides a more personalized prognostic prediction than the AJCC staging systems. In addition, nomograms have better clinical benefits and other researchers have achieved the same results in other oncology studies. Through this novel and easy-to-implement scoring system, personalized survival prognosis predictions after surgery can be easily obtained. Identifying colon cancer patients with different survival risks based on the nomograms may have an impact on further treatment or follow-up plans.

Using the SEER (25) data allows to draw reasonable conclusions consistent with general clinical practice based on a large sample number of colon cancer patients, which is impossible to achieve in a single institutional study. However, this study had some limitations that should be concerning. First, even if the SEER database was regularly checked for discrepancies, it has been reported that its accuracy is 98% and the possibility of incorrect coding or erroneous data still exists. In addition, other potentially prognostic factors including lymphatic vessel invasion, marginal status, surgical procedures, postoperative complications, laboratory indices, and chemotherapy data were not used. More well-known predictors for improving model performance should be applied. Besides, the current study was limited by its retrospective nature, although it was based on a large database. Furthermore, the currents study was based on a Western database of SEER program (25, 35), and further cohorts from Eastern countries are still needed to validate our findings.

Conclusions

In summary, this study developed a prognostic nomogram for patients with non-metastatic colon cancer. The nomogram improves the estimates provided by the current AJCC 8th TNM staging system and can more accurately estimate the survival rate for individual patients after surgery. It might be useful for medical professionals to develop further treatment options and long-term follow-up plans for patients undergoing colon cancer surgery.

Data Availability Statement

The datasets analyzed during this current study are available in SEER database (https://seer.cancer.gov/) to extract the eligible cases. The data are also available from the corresponding author on reasonable request.

Ethics Statement

Ethical approval was not needed as 3rd party data from the SEER database was used.

Author Contributions

J-PP, C-DZ, and D-QD conceived and designed the study. J-PP and YL analyzed the data. J-PP, C-DZ, CZ, and K-ZW wrote the paper. D-QD and Z-MZ reviewed the draft. All authors read and approved the final manuscript.

Funding

This research was funded by the China Scholarship Council, grant number 201908050148 and the Japan China Sasakawa Medical Fellowship, grant number 2017816.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We acknowledge the efforts of the Surveillance, Epidemiology, and End Results (SEER) Program tumor registries for creating the SEER database (https://seer.cancer.gov/).

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fonc.2020.00733/full#supplementary-material

References

1. Siegel RL, Miller KD, Fedewa SA, Ahnen DJ, Meester RG, Barzi A, et al. Colorectal cancer statistics, 2017. Cancer J Clin. (2017) 67:177–93. doi: 10.3322/caac.21395

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Sun X, Suo J, Yan J. Immunotherapy in human colorectal cancer: challenges and prospective. World J Gastroenterol. (2016) 22:6362. doi: 10.3748/wjg.v22.i28.6362

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Wolpin BM, Mayer RJ. Systemic treatment of colorectal cancer. Eur J Cancer. (1993) 134:1296–310.e1. doi: 10.6004/jnccn.2017.0077

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Guyot F, Faivre JS, Meny B, Bonithon KC, Bouvier AM. Time trends in the treatment and survival of recurrences from colorectal cancer. Ann Oncol. (2005) 16:756–61. doi: 10.1093/annonc/mdi151

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Greene FL. Current TNM staging of colorectal cancer. Lancet Oncol. (2007) 8:7. doi: 10.1016/S1470-2045(07)70185-7

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Amin MB, Greene FL, Edge SB, Compton CC, Gershenwald JE, Brookland RK, et al. The eighth edition aJCC cancer staging manual: continuing to build a bridge from a population-based to a more “personalized” approach to cancer staging. Cancer J Clin. (2017) 67:93–9. doi: 10.3322/caac.21388

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Burke HB. Outcome prediction and the future of the tNM staging system. J Natl Cancer Inst. (2004) 96:1408–9. doi: 10.1093/jnci/djh293

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Weiser MR, Landmann RG, Kattan MW, Gonen M, Shia J, Chou J, et al. Individualized prediction of colon cancer recurrence using a nomogram. J Clin Oncol. (2008) 26:380–5. doi: 10.1200/JCO.2007.14.1291

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Compton CC. Colorectal carcinoma: diagnostic, prognostic, and molecular features. Mod Pathol. (2003) 16:376–88. doi: 10.1097/01.MP.0000062859.46942.93

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Mcmillan DC. Systemic inflammation, nutritional status and survival in patients with cancer. Curr Opin Clin Nutr Metab Care. (2009) 12:223–6. doi: 10.1097/MCO.0b013e32832a7902

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Chen L-T, Macarulla T, Blanc J-F, Mirakhur B, Jong FAd, Belanger B, et al. Nomogram for predicting survival in patients treated with liposomal irinotecan plus fluorouracil and leucovorin in metastatic pancreatic cancer. Cancers. (2019) 11:1068. doi: 10.3390/cancers11081068

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Mariani P, Dureau S, Savignoni A, Rouic LL-L, Levy-Gabriel C, Piperno-Neumann S, et al. Development of a prognostic nomogram for liver metastasis of uveal melanoma patients selected by liver mRI. Cancers. (2019) 11:863. doi: 10.3390/cancers11060863

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Chon YE, Park H, Hyun HK, Ha Y, Kim MN, Kim BK, et al. Development of a new nomogram including neutrophil-to-lymphocyte ratio to predict survival in patients with hepatocellular carcinoma undergoing transarterial chemoembolization. Cancers. (2019) 11:509. doi: 10.3390/cancers11040509

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Wang Y, Li J, Xia Y, Gong R, Wang K, Yan Z, et al. Prognostic nomogram for intrahepatic cholangiocarcinoma after partial hepatectomy. J Clin Oncol. (2013) 31:1188–95. doi: 10.1200/JCO.2012.41.5984

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Zheng P, Chen Q, Li J, Jin C, Kang L, Chen D. Prognostic significance of tumor deposits in patients with stage iII colon cancer: a Nomogram study. J Surg Res. (2020) 245:475–82. doi: 10.1016/j.jss.2019.07.099

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Kanemitsu Y, Kato T. Prognostic models for predicting death after hepatectomy in individuals with hepatic metastases from colorectal cancer. World J Surg. (2008) 32:1097–107. doi: 10.1007/s00268-007-9348-0

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Dominique E, Matthieu F, Diane G, Frédéric D, Charles H, Valérie B, et al. A simple tumor load-based nomogram for surgery in patients with colorectal liver and peritoneal metastases. Ann Surg Oncol. (2014) 21:2052–8. doi: 10.1245/s10434-014-3506-z

CrossRef Full Text | Google Scholar

18. Takakura Y, Okajima M, Kanemitsu Y, Kuroda S, Egi H, Hinoi T, et al. External validation of two nomograms for predicting patient survival after hepatic resection for metastatic colorectal cancer. World J Surg. (2011) 35:2275–82. doi: 10.1007/s00268-011-1194-4

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Segelman J, Akre O, Gustafsson U, Bottai M, Martling A. Individualized prediction of risk of metachronous peritoneal carcinomatosis from colorectal cancer. Colorectal Dis. (2014) 16:359–67. doi: 10.1111/codi.12552

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Peng J, Ding Y, Tu S, Shi D, Sun L, Li X, et al. Prognostic nomograms for predicting survival and distant metastases in locally advanced rectal cancers. PLoS ONE. (2014) 9:e106344. doi: 10.1371/journal.pone.0106344

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Vincenzo V, Stiphout RGPM, Guido L, Maria Antonietta G, Maria Cristina B, et al. Nomograms for predicting local recurrence, distant metastases, and overall survival for patients with locally advanced rectal cancer on the basis of european randomized clinical trials. J Clin Oncol. (2011) 29:3163. doi: 10.1200/JCO.2010.33.1595

CrossRef Full Text | Google Scholar

22. Gijn WV, Stiphout RGPMV, Velde CJHVD, Valentini V, Lammering G, Gambacorta MA, et al. Nomograms to predict survival and the risk for developing local or distant recurrence in patients with rectal cancer treated with optional short-term radiotherapy. Ann Oncol. (2015) 26:928. doi: 10.1093/annonc/mdv023

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Beppu T, Sakamoto Y, Hasegawa K, Honda G, Tanaka K, Kotera Y, et al. A nomogram predicting disease-free survival in patients with colorectal liver metastases treated with hepatic resection: multicenter data collection as a project study for hepatic surgery of the japanese society of hepato-Biliary-Pancreatic surgery. J Hepato-Bil Pancr Sci. (2012) 19:72–84. doi: 10.1007/s00534-011-0460-z

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Greene FL, Page DL, Fleming ID, Fritz A, Balch CM, Haller DG, et al. American Joint Committee on Cancer (AJCC) Cancer Staging Manual. 6th edn. New York, NY: Springer (2002).

Google Scholar

25. Surveillance Epidemiology and End Results (SEER) Program. (www.seer.cancer.gov) SEER*Stat Database: Incidence — SEER 18 Regs Custom Data (With Additional Treatment Fields). Nov 2017 Sub (1973–2015 varying) — Linked To County Attributes — Total U.S., 1969–2016 Counties, National Cancer Institute, DCCPS, Surveillance Research Program (2018).

26. Akaike H. A new look at the statistical model identification. Autom Control Trans. (1974) 19:716–23. doi: 10.1109/TAC.1974.1100705

CrossRef Full Text | Google Scholar

27. Alba AC, Agoritsas T, Walsh M, Hanna S, Iorio A, Devereaux PJ, et al. Discrimination and calibration of clinical prediction models: users' guides to the medical literature. JAMA. (2017) 318:1377–84. doi: 10.1001/jama.2017.12126

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Deci Making. (2006) 26:565–74. doi: 10.1177/0272989X06295361

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Fitzgerald M, Saville BR, Lewis RJ. Decision curve analysis. JAMA. (2015) 313:409–10. doi: 10.1001/jama.2015.37

PubMed Abstract | CrossRef Full Text | Google Scholar

30. O'Connell JB, Maggard MA, Ko CY. Colon cancer survival rates with the new american joint committee on cancer sixth edition staging. J Natl Cancer Inst. (2004) 96:1420–5. doi: 10.1093/jnci/djh275

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Edge SB, Byrd DR, Fritz AG, Greene FL, Trotti A. AJCC Cancer Staging Manual. 7th edn. New York, NY: Springer (2010).

Google Scholar

32. Betge J, Harbaum L, Pollheimer MJ, Lindtner RA, Kornprat P, Ebert MP, et al. Lymph node retrieval in colorectal cancer: determining factors and prognostic significance. Int J Colorectal Dis. (2017) 32:991–8. doi: 10.1007/s00384-017-2778-8

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Wong SL, Ji H, Hollenbeck BK, Morris AM, Baser O, Birkmeyer JD. Hospital lymph node examination rates and survival after resection for colon cancer. JAMA. (2007) 298:2149–54. doi: 10.1001/jama.298.18.2149

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Baxter NN, Virnig DJ, Rothenberger DA, Morris AM, Jessurun J, Virnig BA. Lymph node evaluation in colorectal cancer patients: a population-based study. J Natl Cancer Inst. (2005) 97:219–25. doi: 10.1093/jnci/dji020

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Pei JP, Zhang CD, Fan YC, Dai DQ. Comparison of different lymph node staging systems in patients with resectable colorectal cancer. Front Oncol. (2018) 8:671. doi: 10.3389/fonc.2018.00671

PubMed Abstract | CrossRef Full Text | Google Scholar