Number of Positive Lymph Nodes Is Superior to LNR and LODDS for Predicting the Prognosis of Pancreatic Neuroendocrine Neoplasms

Background: The American Joint Committee on Cancer (AJCC) staging for pancreatic neuroendocrine neoplasms (PanNENs) based on the number of positive lymph nodes (PLNs) is the most widely accepted nodal staging system. New nodal staging schemes that take both the number of PLNs and the number of examined lymph nodes into consideration have emerged as useful prognostic tools. The aim of the current study was to determine the most effective nodal staging system, among the 8th edition AJCC N staging (or PLN staging), lymph node ratio (LNR), and log odds of positive lymph nodes (LODDS), for predicting the cause-specific survival of patients with PanNENs.

Methods: The clinicopathological and prognostic data of 2,295 patients from the Surveillance, Epidemiology, and End Results (SEER) database, diagnosed with PanNENs between 1988 and 2015, were reviewed retrospectively.

Results: A multivariate analysis identified PLN and LNR staging as independent prognostic factors, but not LODDS. The PLN staging exhibited higher C-index and area under the curve values than those of the LNR and LODDS, indicating better predictive discriminatory capacity. No significant difference in the survival of patients was observed within the same PLN staging subgroup according to the number (high or low) of examined lymph nodes. In contrast, intra-group heterogeneity was seen with use of LNR and LODDS staging, due to overestimation of the risk of insufficient examined lymph nodes, and LODDS failed to stratify patients without lymph nodes metastasis into different risk groups.

Conclusions: The PLN staging is more reliable than LNR and LODDS staging for predicting the cause-specific survival of PanNENs.

Introduction

Pancreatic neuroendocrine neoplasms (PanNENs) are malignancies arising from the pancreas showing neuroendocrine differentiation (1). Their incidence has significantly increased over the past few decades, probably due to advancements in imaging techniques (2, 3). PanNENs exhibit heterogeneous biological behavior, and include indolent and aggressive types. Based on differences in the mitotic count and Ki-67 labeling index, PanNENs can be divided into well-differentiated pancreatic neuroendocrine tumors (PanNETs) and poorly differentiated pancreatic neuroendocrine carcinomas (PanNECs) (1). In the 8th edition of the American Joint Committee on Cancer (AJCC) Cancer Staging Manual, the nodal staging system for PanNETs is based only on nodal status (negative or positive), while the system for PanNECs is based on the number of positive lymph nodes (PLNs) (4). Although it is challenging to stratify patients into different risk groups due to the rarity and heterogeneity of PanNENs, a unified staging scheme is more attractive in clinical application. Partelli et al. reported that the number of PLNs accurately predicts recurrence not only for PanNECs but also for PanNETs (5). New nodal staging schemes that take both the number of PLNs and the number of examined lymph nodes (or negative lymph nodes) into consideration have emerged as useful prognostic tools. The lymph node ratio (LNR) is defined as the ratio between the number of PLNs and the total number of examined lymph nodes. The LNR is being increasingly recognized as a strong predictor of survival in patients with other pancreatic neoplasms, including pancreatic adenocarcinoma (6) and intraductal papillary mucinous neoplasms (7). Previously, it was reported that an LNR-based staging system is superior to the current 8th edition AJCC staging system for PanNENs (8). Log odds of positive lymph nodes (LODDS) is defined as log (PLNs+0.5)/(examined lymph nodes-PLNs+0.5). Adding 0.5 to the numerator and denominator avoids division by zero, which aids in stratification of patients without PLNs into different prognostic groups (9). Recently, LODDS staging was thought a better predictor of survival in the small bowel neuroendocrine tumors compared with LNR and PLN staging (10). However, the actual prognostic value of LODDS in PanNENs remains unclear. Therefore, the aim of the current study was to determine the most appropriate nodal staging system for predicting cause-specific survival in PanNENs, by evaluating the prognostic ability of the PLN (or 8th edition AJCC N staging), LNR, and LODDS staging systems.

Materials and Methods

The data of patients with PanNENs during the period 1988 to 2015 were obtained from the Surveillance, Epidemiology, and End Results (SEER) database. The following histological subtypes were included in the analysis, based on their International Classification of Diseases for Oncology, Third Edition codes: islet-cell adenocarcinoma (8150), malignant beta-cell tumor (8151), malignant alpha-cell tumor (8152), G-cell tumor (8153), mixed islet cell and exocrine tumor (8154), VIPoma (8155), malignant somatostatinoma (8156), carcinoid tumor (8240), enterochromaffin cell tumor(8241), goblet carcinoid tumor (8243), mixed adenoneuroendocrine carcinoma (8244), neuroendocrine carcinoid (8246), and atypical carcinoid tumor (8249).

The patient eligibility criteria were as follows: (1) histologically confirmed neuroendocrine tumors; (2) primary tumor located in the pancreas; (3) no other malignant tumors; (4) ≥18 years of age; (5) treatment by primary site surgery; (6) at least one lymph node was examined; (7) complete PLN data; (8) survival time of more than 0 months; and (9) close follow-up. In this study, the primary outcome was cause-specific survival, defined as death caused by PanNENs.

Scatterplots and Spearman’s correlation coefficients were used to evaluate the distributions of, and correlations among, continuous PLN, LNR, and LODDS variables. Smooth curves from restricted cubic spline were plotted to assess the non-linear relationship between three nodal staging systems and log hazard ratio (HR) by univariate Cox regression model (11). X-tile software (version 3.6.1; https://medicine.yale.edu/lab/rimm/research/software/) was used to determine the optimal cutoff points for the PLN, LNR, and LODDS staging systems. This software divides patients into two or three groups based on the χ2 value. Cross-validation was performed to derive corrected P-values (12). The discriminative efficacy of the different nodal staging systems was assessed by Harrell’s concordance index (C-index), which ranges from 0.5 (no predictive power) to 1 (complete differentiation) (13). Because the survival rate of the patients changed with time, we also used time-dependent area under the receiver operating characteristic curve (AUC) analysis to examine the discriminative abilities of the different nodal staging systems over a 10-year period (14). Univariate and multivariate analyses were performed to evaluate the prognostic performance of the three nodal staging systems and other possible prognostic factors. All factors with a P-value <0.2 in the univariate analysis were included in Cox proportional hazards multivariate models. Cases with missing data were excluded from the multivariate analysis.

All analyses were conducted using X-tile (version 3.6.1; Yale University School of Medicine, New Haven, CT, USA), SPSS (version 22.0; SPSS Inc., Chicago, IL, USA), and R software (version 3.5.3; R Foundation for Statistical Computing, Vienna, Austria). A P-value <0.05 was considered significant, and all tests were two-tailed.

Results

Characteristics of Patients

A total of 2,295 patients with PanNENs who met the inclusion criteria were enrolled in this study (Figure 1); their clinical characteristics are described in Table 1. The study population was 78.7% white, 12.0% black, and 9.3% other races. About 53% of the patients were male and 47% were female. The median age at diagnosis was 57 years. About 90.0% of patients underwent a local or partial pancreatectomy. Most tumors were located in the tail of the pancreas (40.2%) or the head of the pancreas (34.4%). The vast majority of tumors were non-functioning (97.4%) and well-to moderately differentiated on histological examination (91.6%). The results of the univariate analysis showed that age, sex, marriage, extent of resection, histologic grade, T staging, M staging, and the number of examined lymph nodes had significant prognostic value (P < 0.05).

FIGURE 1

Figure 1 Flow diagram of patient selection.

TABLE 1

Table 1 Clinicopathologic characteristics of patients.

Characteristics of Three Nodal Staging Schemes

Scatterplots (Figure 2) were created to assess the relationships of PLN, LNR, and LODDS data. Each PLN value could be divided into different LNR and LODDS values. Each LODDS value had a one-to-one correspondence with an LNR value, except when LNR equaled 0 or 1. The LODDS data were more strongly correlated with the LNR than PLN data (r = 0.752 vs. r = 0.703).

FIGURE 2

Figure 2 Distribution characteristics of LODDS and PLN (A), LNR and PLN (B), LODDS and LNR (C).

Restricted cubic splines (Figure 3) were used to assess the relationships between the three lymph node staging systems and log HR. A nonlinear association was found, and the mortality risk increased as the LODDS, LNR, and PLN values increased in all patients with PanNENs. Figure 3D showed that the confidence interval (CI) included 0, suggesting that LODDS was neither a risk factor nor a protective factor in node-negative patients.

FIGURE 3

Figure 3 Association between PLN (A), LNR (B), and LODDS (C) and log hazard ratio in all patients and association between LODDS (D) and log hazard ratio in the patients without positive lymph nodes involvement. The solid line and the red dashed lines represent the estimated log hazard ratio and the 95% confidence interval, respectively.

Three-Category Nodal Staging Schemes

To compare the three lymph node staging schemes from three-category perspective, the continuous variables of the PLN, LNR, and LODDS were divided into three groups using X-Tile software. The PLN staging system includes three groups: PLN1(= 0), n = 1,425 (62.1%); PLN2 (1–3), n = 570 (24.8%); and PLN3 (≥4), n = 300 (13.1%). It happens to coincide with the 8th edition AJCC N staging for the PanNECs. The LNR also yielded three risk groups: LNR1 (= 0), n = 1, 425 (62.1%); LNR2 (0–0.28), n = 474 (20.7%); and LNR3 (>0.28), n = 396 (17.3%). Finally, the LODDS staging consisted of the following three groups: LODDS1 (≤−2.56), n = 961 (41.9%); LODDS2 (−2.56 to −0.66), n = 968 (42.2%); and LODDS3 (>−0.66), n = 366 (15.9%) (Table 2).

TABLE 2

Table 2 Univariate and multivariate analysis of three-category nodal staging schemes.

The results of the univariate analysis showed that the three-category PLN, LNR, and LODDS staging systems all had significant prognostic value. However, in the multivariate analysis, only PLN and LNR staging were independent prognostic factors of a worse survival (Table 2).

Based on the C-index value, PLN staging (0.642; 95% CI: 0.611–0.673) had higher prognostic value compared with LNR staging (0.636; 95% CI: 0.606–0.666) and LODDS staging (0.617; 95% CI: 0.585–0.648). As shown in Figure 4, PLN staging consistently had higher AUC values at the 1-, 2-, 3-, 4-, 5-, 6-, 7-, 8-, 9-, and 10-year follow-ups than LNR or LODDS staging in the survival analysis, suggesting that it was better able to distinguish between the prognosis of patients with PanNENs, consistent with the C-index result.

FIGURE 4

Figure 4 Time-dependent area under the receiver operating characteristic curve (AUC) in patients with PanNENs for three-category staging schemes.

As shown in Table 3, we then compared the survival rates of patients staged using the PLN, LNR, and LODDS staging systems according to the number of examined lymph nodes. For patients in each PLN staging subgroup (PLN1, PLN2, and PLN3), the survival was highly homogenous among different groups of examined lymph nodes (all P^a > 0.05). For patients in LNR3, LODDS2, and LODDS3 subgroups, statistical differences in the survival could be observed among different groups of examined lymph nodes $\left({\text{P}}_{\text{LNR}3}^{\text{a}}=0.035,{\text{P}}_{\text{LODDS}2}^{\text{a}}=0.001,{\text{P}}_{\text{LODDS}3}^{\text{a}}=0.025\right)$. To find out why LNR and LODDS systems, rather than PLN, showed intra-group heterogeneity, the prognoses of patients with 1–5, 6–10, 11–15, and 16–20 examined lymph nodes were compared with those with >20 examined lymph nodes respectively. In each PLN staging subgroup (PLN1, PLN2, and PLN3), the prognoses of patients with ≤ 20 examined lymph nodes were similar to those of patients with >20 examined lymph nodes (all p^b > 0.05), suggesting that an insufficient number of examined lymph nodes would not lead to PLN staging migration. By contrast, the ratio lymph nodal staging schemes (LNR and LODDS) overestimated the HR for patients with insufficient number of examined lymph nodes. For example, when the number of examined lymph nodes was between 1 and 5, patients were assigned only to the LODDS2 and LODDS3 subgroups, and not to LODDS1. However, patients with 1–5 examined lymph nodes had a significantly better prognosis than those with >20 examined lymph nodes in the LODDS2 and LODDS3 subgroups (both P^b < 0.001).

TABLE 3

Table 3 Survival analysis on the basis of three-category nodal staging schemes according to the number of examined lymph nodes (ELNs).

LODDS was said to have an inherent advantage for assessing node-negative patients. However, in our study, node-negative patients were assigned only to LODDS1 and LODDS2, and not to LODDS3. The survival curves of node-negative patients stratified by LODDS staging are plotted in Figure 5. No significant difference was observed in survival probability between the LODDS1 and LODDS2 subgroups (P > 0.05), indicating that three-category LODDS staging could not stratify node-negative patients into different risk groups.

FIGURE 5

Figure 5 Disease-specific survival curves of PanNENs patients without positive lymph nodes according to three-category LODDS staging (log rank P = 0.17).

Two-Category Nodal Staging Schemes

We compared the different lymph node staging schemes from the perspective of two categories. Using X-Tile software, two-stage solutions were derived for the PLN, LNR, and LODDS staging systems, as follows. For the PLN staging system, PLN1 (= 0), n = 1,425 (62.1%); and PLN2 (>0), n = 870 (37.9%), consistent with the 8th edition AJCC N staging system for the neuroendocrine pancreas. For the LNR staging system, LNR1 (= 0), n = 1,425 (62.1%); and LNR2 (>0), n = 870 (37.9%). For the LODDS staging system, LODDS1 (≤ −0.93), n = 1,869 (81.4%); and LODDS2 (>−0.93), n = 426 (18.6%) (Table 4). Only the PLN and LODDS staging systems are discussed in the following section, as the LNR and PLN systems were not different. The result was similar to that of three-category staging schemes.

TABLE 4

Table 4 Univariate and multivariate analysis of two-category nodal staging schemes.

The two-category PLN and LODDS staging were significantly correlated with prognosis in the univariate analysis. However, only the two-category PLN staging was an independent prognostic factor in the multivariate analysis (Table 4).

As measured by Harrell’s C-index, PLN staging (0.623; 95% CI: 0.595–0.651) had better discriminatory capacity than LODDS staging (0.591; 95% CI: 0.564–0.619). As shown in Figure 6, PLN staging had higher 1-, 2-, 3-, 4-, 5-, 6-, 7-, 8-, 9-, and 10-year AUC values than LODDS staging.

FIGURE 6

Figure 6 Time-dependent area under the receiver operating characteristic curve (AUC) for two-category staging schemes in patients with PanNENs.

We compared the survival rates of patients using the PLN and LODDS staging systems according to the number of examined lymph nodes. As shown in Table 5, for patients in each PLN staging subgroup (PLN1 and PLN2), the survival was highly homogenous among different groups of examined lymph nodes $\left({\text{P}}_{\text{PLN}1}^{\text{a}}=0.371,{\text{P}}_{\text{PLN}2}^{\text{a}}=0.249\right)$. However, significant differences in survival were observed in the LODDS1 and LODDS2 subgroups $\left({\text{P}}_{\text{LODDS}1}^{\text{a}}=0.013,{\text{P}}_{\text{LODDS}2}^{\text{a}}=0.008\right)$, indicating that LODDS staging had increased intra-group heterogeneity compared with PLN staging.

TABLE 5

Table 5 Survival analysis on the basis of two-category nodal staging schemes according to the number of examined lymph nodes (ELNs).

Patients were assigned only to the LODDS1 subgroup when the number of PLNs was zero. Therefore, the two-category LODDS staging failed to stratify node-negative patients into distinct survival groups.

Discussion

Lymph node metastasis was thought to be an independent predictor of poor outcomes in patients with PanNENs (15, 16). AJCC staging based on the number of PLNs is the most widely accepted nodal staging system. The staging algorithm for PanNENs is the same as that for pancreatic carcinomas in the 7th edition AJCC cancer staging manual (17). A large retrospective cohort study reported that European Neuroendocrine Tumor Society (ENETS) system, which was specifically designed for staging PanNENs, better distinguishes among stages and has higher predictive ability than the 7th edition AJCC system (18, 19). Therefore, the 8th edition AJCC staging was modified as follows: well-differentiated PanNETs are now staged using the staging system for the neuroendocrine pancreas (which is consistent with the ENETS system), while poorly differentiated PanNECs are still staged using the system for pancreatic carcinomas. The 8th edition AJCC N staging for PanNETs is a two-category system (N0: no metastasis; N1: ≥1 metastatic lymph nodes); the 8th edition AJCC N staging for PanNECs is a three-category system (N0: no metastasis; N1: 1–3 metastatic lymph nodes; N2: ≥4 metastatic lymph nodes) (9). Based on the number of PLNs, we used X-tile software to divide the PanNENs patients in the SEER database into two populations (0, and >0), and then into three populations (0, 1–3, and ≥4). It was interesting to note that these divisions were consistent with the AJCC N staging system adopted for PanNETs and PanNECs, indicating that the cut-off points in the current 8th edition AJCC N staging system are reasonable. In addition, our study demonstrated that three-category nodal staging had better discriminatory capacity than the two-category system to predict cause-specific survival of PanNENs, as measured by the C-index and time-independent AUC. This conclusion is similar to that of a previous small retrospective study, which proposed that three-category nodal staging is accurate for predicting the recurrence of both PanNETs and PanNECs (5).

The PLN staging (or 8th edition AJCC N staging) depends only on the number of PLNs; the impact of the number of examined lymph nodes (or negative lymph nodes) is not considered. When the number of PLNs is the same, patients with an insufficient number of examined lymph nodes might have a poorer prognosis; this phenomenon of understaging is called staging migration or Will Rogers phenomenon (20).

The LNR staging system may reduce the risk of the staging migration in some cancers, such as gastric cancers (21). However, the utility of the LNR is limited when the ratio value is zero. Another drawback is that patients with the same LNR count (e.g., 1/1 vs. 30/30) may have different prognoses. The optimal LNR cutoff in PanNENs is controversial and varies among studies, perhaps due to differences in the end points or number of patients. Two small retrospective studies assessed the LNR using cutoffs of 0.2 and 0.07, and found that it was a significant predictor of recurrence in patients with PanNENs (22, 23). Using SEER data, Liu et al. showed that an LNR > 0.40 was a significant adverse prognostic factor with respect to overall survival (24). Gaitanidis et al. reported that an LNR ≥ 0.5 was independently associated with a worse cause-specific survival and the LNR-based staging system was superior to 8th edition AJCC N staging (8). In our study, which contained the largest number of PanNENs patients among all of these studies, LNR staging predicted cause-specific survival in patients with PanNENs, but did not show higher prognostic utility than the AJCC N staging system.

LODDS is a new approach to evaluate nodal status. LODDS comprehensively considers the numbers of PLNs and negative lymph nodes, and thus can also reduce the likelihood of staging migration due to examination of an insufficient number of lymph nodes. In addition, it overcomes the major shortcoming of the LNR: the LNR value does not increase with an increase in the number of PLNs when all examined lymph nodes are positive. Besides, the LODDS value decreases with an increase in the number of negative lymph nodes when all examined lymph nodes are negative. Therefore, LODDS has a unique value for risk stratification of the node-negative patients, which LNR and PLN staging systems do not possess (25). However, the practical utility of LODDS remains controversial. For example, Morales-Oyarvide et al. found no obvious improvement in prognostic value when using the LNR and LODDS classifications compared with the PLN classification in pancreatic ductal adenocarcinomas (25). To the best of our knowledge, the prognostic role of LODDS has never been analyzed in PanNENs. Our research demonstrates that LODDS fails to discriminate node-negative patients in terms of survival. In the two-category nodal staging, the node-negative patients were not reclassified into different LODDS subgroups; they were all assigned to the LODDS1 subgroup. In the three-category nodal staging, the node-negative patients could be reclassified LODDS2 and LODDS3 subgroups, but the difference in survival between them was not significant. In addition, restricted cubic spline functions indicate that LODDS is neither a risk factor nor a protective factor when the number of PLNs is zero.

The current study demonstrates that PLN staging performs better than LNR and LODDS staging for predicting cause-specific survival in patients with PanNENs after pancreatic resection surgery from both two-category and three-category perspective. Firstly, PLN and LNR staging, but not LODDS staging, were independent prognostic factors in the multivariate analysis. Secondly, PLN staging showed greater C-index and AUC values than the LNR and LODDS staging systems, corresponding to better discriminatory capacity. Thirdly, and most importantly, no significant differences in survival were observed among patients in the same PLN staging subgroup according to the examined lymph node value (high or low). On the contrary, LNR and LODDS systems showed intra-group heterogeneity due to overestimating the risk of an insufficient number of examined lymph nodes. In consideration of the relatively good prognosis and limited prognostic value of lymph node metastasis in the patients with PanNENs, the ratio lymph nodal staging schemes (LNR and LODDS) designed to reduce intra-group heterogeneity was actually counterproductive.

The current study had some inherent limitations associated with the inherent limitations of the SEER database. For example, the SEER database does not include several important prognostic factors, such as patients’ symptoms, margin status, and chemotherapy, which may have affected the results of our Cox multivariate analysis. In addition, we excluded patients with missing data on the number of positive lymph nodes and examined lymph nodes, which may have caused selection bias. However, these are common shortcomings of retrospective and population-based studies. Given the rarity of PanNENs, retrospective studies based on large cancer registries are still of great significance, and prospective studies are needed to validate our conclusions.

In conclusion, the predictive value of the PLN staging system was not affected by an insufficient number of examined lymph nodes, while use of the LNR and LODDS systems was associated with an increased risk of nodal staging migration. The LODDS system did not stratify node-negative patients in terms of risk. Currently, the PLN staging (or 8th edition AJCC N staging) is superior to the LNR and LODDS staging.

Data Availability Statement

Publicly available datasets were analyzed in this study. This data can be found here: https://seer.cancer.gov/data/.

Author Contributions

BG and DZ conceived the study. BG, ZL, XQ, WZ, and YJ collected and analyzed data. BG, DZ, and ZL wrote the manuscript. WW revised it critically. All authors contributed to the article and approved the submitted version.

Funding

This work was supported by the National Natural Science Foundation of China (no. 81572307 and 81773096) and the Major Project of Medical and Health Technology Development Program in Zhejiang Province (No. 7211902).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

1. Lloyd RV, Osamura RY, Klöppel G, Rosai J. WHO Classification of Tumours of Endocrine Organs. 4th edn. Lyon: IARC Press (2017).

Google Scholar

2. Hallet J, Law CH, Cukier M, Saskin R, Liu N, Singh S. Exploring the rising incidence of neuroendocrine tumors: a population-based analysis of epidemiology, metastatic presentation, and outcomes. Cancer (2015) 121(4):589–97. doi: 10.1002/cncr.29099

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Gordon-Dseagu VL, Devesa SS, Goggins M, Stolzenberg-Solomon R. Pancreatic cancer incidence trends: evidence from the Surveillance, Epidemiology and End Results (SEER) population-based data. Int J Epidemiol (2018) 47(2):427–39. doi: 10.1093/ije/dyx232

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Amin MB, Edge SB. AJCC cancer staging manual. 8th ed. New York (NY): Springer (2017).

Google Scholar

5. Partelli S, Javed AA, Andreasi V, He J, Muffatti F, MJ W, et al. The number of positive nodes accurately predicts recurrence after pancreaticoduodenectomy for nonfunctioning neuroendocrine neoplasms. Eur J Surg Oncol (2018) 44(6):778–83. doi: 10.1016/j.ejso.2018.03.005

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Slidell MB, Chang DC, Cameron JL, Wolfgang C, Herman JM, Schulick RD, et al. Impact of total lymph node count and lymph node ratio on staging and survival after pancreatectomy for pancreatic adenocarcinoma: a large, population-based analysis. Ann Surg Oncol (2008) 15(1):165–74. doi: 10.1245/s10434-007-9587-1

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Partelli S, Fernandez-Del Castillo C, Bassi C, Mantovani W, Thayer SP, Crippa S, et al. Invasive intraductal papillary mucinous carcinomas of the pancreas: predictors of survival and the role of lymph node ratio. Ann Surg (2010) 251(3):477–82. doi: 10.1097/SLA.0b013e3181cf9155

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Gaitanidis A, Patel D, Nilubol N, Tirosh A, Kebebew E. A Lymph Node Ratio-Based Staging Model Is Superior to the Current Staging System for Pancreatic Neuroendocrine Tumors. J Clin Endocrinol Metab (2018) 103(1):187–95. doi: 10.1210/jc.2017-01791

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Vinh-Hung V, Verschraegen C, Promish DI, Cserni G, Van de Steene J, Tai P, et al. Ratios of involved nodes in early breast cancer. Breast Cancer Res (2004) 6(6):R680–8. doi: 10.1210/jc.2017-01791

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Jiang SJ, Zhao LH, Xie CY, Su HF, Yan Y. Prognostic performance of different lymph node staging systems in patients with small bowel neuroendocrine Tumors. Front Endocrinol (Lausanne) (2020) 11:402. doi: 10.3389/fendo.2020.00402

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Heinzl H, Kaider A. Gaining more flexibility in Cox proportional hazards regression models with cubic spline functions. Comput Methods Programs Biomed (1997) 54(3):201–8. doi: 10.1016/s0169-2607(97)00043-6

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Camp RL, Dolled-Filhart M, Rimm DL. X-tile: a new bio-informatics tool for biomarker assessment and outcome-based cut-point optimization. Clin Cancer Res (2004) 10(21):7252–9. doi: 10.1158/1078-0432.CCR-04-0713

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med (1996) 15(4):361–87. doi: 10.1002/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Kamarudin AN, Cox T, Kolamunnage-Dona R. Time-dependent ROC curve analysis in medical research: current methods and applications. BMC Med Res Methodol (2017) 17(1):53. doi: 10.1186/s12874-017-0332-6

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Hashim YM, Trinkaus KM, Linehan DC, Strasberg SS, Fields RC, Cao D, et al. Regional lymphadenectomy is indicated in the surgical treatment of pancreatic neuroendocrine tumors (PNETs). Ann Surg (2014) 259(2):197–203. doi: 10.1097/SLA.0000000000000348

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Tanaka M, Heckler M, Mihaljevic AL, Probst P, Klaiber U, Heger U, et al. Systematic review and metaanalysis of lymph node metastases of resected pancreatic neuroendocrine tumors. Ann Surg Oncol (2020). doi: 10.1245/s10434-020-08850-7

CrossRef Full Text | Google Scholar

17. Edge SB, Byrd DR, Compton CC. AJCC Cancer Staging Manual. 7th ed. New York (NY: Springer (2010).

Google Scholar

18. Rindi G, Kloppel G, Alhman H, Caplin M, Couvelard A, de Herder WW, et al. TNM staging of foregut (neuro)endocrine tumors: a consensus proposal including a grading system. Virchows Arch (2006) 449(4):395–401. doi: 10.1007/s00428-006-0250-1

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Rindi G, Falconi M, Klersy C, Albarello L, Boninsegna L, Buchler MW, et al. TNM staging of neoplasms of the endocrine pancreas: results from a large international cohort study. J Natl Cancer Inst (2012) 104(10):764–77. doi: 10.1093/jnci/djs208

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Feinstein AR, Sosin DM, Wells CK. The Will Rogers phenomenon. Stage migration and new diagnostic techniques as a source of misleading statistics for survival in cancer. N Engl J Med (1985) 312(25):1604–8. doi: 10.1056/NEJM198506203122504

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Sun Z, Zhu GL, Lu C, Guo PT, Huang BJ, Li K, et al. The impact of N-ratio in minimizing stage migration phenomenon in gastric cancer patients with insufficient number or level of lymph node retrieved: results from a Chinese mono-institutional study in 2159 patients. Ann Oncol (2009) 20(5):897–905. doi: 10.1093/annonc/mdn707

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Boninsegna L, Panzuto F, Partelli S, Capelli P, Delle Fave G, Bettini R, et al. Malignant pancreatic neuroendocrine tumour: lymph node ratio and Ki67 are predictors of recurrence after curative resections. Eur J Cancer (2012) 48(11):1608–15. doi: 10.1016/j.ejca.2011.10.030

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Ricci C, Casadei R, Taffurelli G, Buscemi S, D’Ambra M, Monari F, et al. The role of lymph node ratio in recurrence after curative surgery for pancreatic endocrine tumours. Pancreatology (2013) 13(6):589–93. doi: 10.1016/j.pan.2013.09.001

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Liu P, Zhang X, Shang Y, Lu L, Cao F, Sun M, et al. Lymph node ratio, but not the total number of examined lymph nodes or lymph node metastasis, is a predictor of overall survival for pancreatic neuroendocrine neoplasms after surgical resection. Oncotarget (2017) 8(51):89245–55. doi: 10.18632/oncotarget.19184

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Morales-Oyarvide V, Rubinson DA, Dunne RF, Kozak MM, Bui JL, Yuan C, et al. Lymph node metastases in resected pancreatic ductal adenocarcinoma: predictors of disease recurrence and survival. Br J Cancer (2017) 117(12):1874–82. doi: 10.1038/bjc.2017.349

PubMed Abstract | CrossRef Full Text | Google Scholar