A Radiomics Nomogram for the Preoperative Prediction of Lymph Node Metastasis in Pancreatic Ductal Adenocarcinoma

Purpose: To construct and verify a CT-based multidimensional nomogram for the evaluation of lymph node (LN) status in pancreatic ductal adenocarcinoma (PDAC).

Materials and Methods: We retrospectively assessed data from 172 patients with clinicopathologically confirmed PDAC surgically resected between February 2014 and November 2016. Patients were assigned to either a training cohort (n = 121) or a validation cohort (n = 51). We acquired radiomics features from the preoperative venous phase (VP) CT images. The maximum relevance–minimum redundancy (mRMR) algorithm and the least absolute shrinkage and selection operator (LASSO) methods were used to select the optimal features. We used multivariable logistic regression to construct a combined radiomics model for visualization in the form of a nomogram. Performance of the nomogram was evaluated by the receiver operating characteristic (ROC) curve approach, calibration testing, and analysis of clinical usefulness.

Results: A Rad score consisting of 10 LN status-related radiomics features was found to be significantly associated with the actual LN status (P < 0.01). A nomogram that consisted of Rad scores, CT-reported parenchymal atrophy, and CT-reported LN status performed well in terms of predictive power in the training cohort (area under the curve, 0.92), and this was confirmed in the validation cohort (area under the curve, 0.95). The nomogram also performed well in the calibration test and decision curve analysis, demonstrating its potential clinical value.

Conclusion: A multidimensional radiomics nomogram consisting of Rad scores, CT-reported parenchymal atrophy, and CT-reported LN status may contribute to the non-invasive evaluation of LN status in PDAC patients.

Introduction

Pancreatic ductal adenocarcinoma (PDAC) is notorious for its occult onset and early metastasis. As one of the several top causes of cancer deaths, the 5-year survival rate of PDAC patients is only 7–8% (1, 2). Early radical surgery is the main treatment modality for patients with PDAC. However, owing to the visceral location of the pancreas and the non-specific symptoms in most early PDAC patients, it is extremely difficult to make an early diagnosis of this disease. This results in limited and suboptimal treatment options for most patients (3). With the intensive efforts to develop neoadjuvant chemotherapy and other new therapeutic methods, there is a growing demand for accurate preoperative staging and personalized tailoring of the therapeutic approach in PDAC. PDAC is well known to be accompanied by the occurrence of lymph node metastasis (LNM), with an LNM rate as high as 59% (4). As an important postoperative prognostic factor, the cancer-positive lymph node (LN) is strongly related to poor prognosis in PDAC patients (5–7). Consequently, there is an urgent need to develop a capability to predict LN status precisely before surgery. Currently, the preoperative status of PDAC patients is mainly evaluated by imaging methods such as CT and MRI. Only relatively poor accuracy can be achieved when evaluating the LN status solely from a morphological perspective (for example, by assessing changes in lymph node size, morphology, and intensity). These approaches are not able to provide effective guidance for clinical treatment and are far from satisfactory predictive factors.

Contrast-enhanced CT (CECT) has long been the preferred imaging modality for preoperative staging of PDAC (8, 9) because it facilitates the assessment of tumor size and vascular involvement. Enlarged lymph nodes as indicated by CECT carry a high positive value for predicting outcome in many malignant tumors, and surgeons can select appropriate LN dissection methods based on the CT report (10–12). For PDAC patients, however, due to the complexity of the peripancreatic structures, it is not easy to define whether LNs are abnormally enlarged. Further complications result from the fact that similar enlarged LNs also appear in local inflammation or secondary biliary obstruction, which can confound the judgment of LN involvement in PDAC. Considering the above factors, CT in fact achieves only a mediocre diagnostic performance for LN metastasis in PDAC, especially regarding its sensitivity (13). MRI and positron emission tomography (PET) have also been considered as potentially useful LNM markers, achieving results similar to those of CT in PDAC patients (14). Recently, the use of endoscopic ultrasound-guided fine needle aspiration (EUS-FNA) has been expanding rapidly for the evaluation of pancreatic masses (15–18). However, EUS-FNA is also affected by many particular complexities, including the investigators’ degree of knowledge of cytopathology, the endosonography technique employed, and the locations and characteristics of the accessed lesions (19, 20).

As an emerging discipline that has attracted numerous researchers’ interests, radiomics extracts multidimensional features contained in available images with high-throughput methods and explores their underlying associations with pathophysiological changes. Recently, several investigators have constructed radiomics models for preoperative LN evaluation in certain gastrointestinal cancers and have succeeded in achieving the desired level of predictive accuracy (21–23). However, to the best of our knowledge, thus far, there are few studies on the development of a radiomics nomogram to predict the LN status for patients with PDAC. To this end, we sought to build and verify a radiomics-based nomogram that could potentially assist in clinical decision-making processes for patients with PDAC.

Materials and Methods

Patient Population

Our retrospective study was approved by the Institutional Review Board of Huashan Hospital, with informed consent waived. The PDAC patients who elected to undergo tumor resection and LN dissection between February 2014 and November 2016 in our hospital were retrospectively evaluated. The inclusion criteria were (1) PDAC patients with histological confirmation; (2) thin-layer CECT performed within 1 month before surgery; (3) patients without previous radiotherapy, surgery, and/or chemotherapy; and (4) patients who underwent pancreaticoduodenectomy and where pathologically evidence of LN status was available.

The exclusion criteria were (1) difficulties in distinguishing the tumors on CT images owing to artifacts or for any other reason; (2) features that could not be successfully extracted from the CT images of the patients; and (3) patients with other coexisting primary malignancies. The detailed selection steps for the patients with PDAC are depicted in Supplementary Figure S1.

Ultimately, 172 patients who met the above criteria were included in this retrospective study. Of these, 121 patients were assigned to the training cohort (64 men and 57 women), with an average age of 63.5 ± 9.2 years (range, 35–83). Another 51 patients (30 men, 21 women) with a mean age of 63.7 ± 8.5 years (range, 45–84) constituted the validation cohort.

Clinical data (for example, age, primary tumor site, and preoperative CA-199 level) were obtained from the medical records. Two radiologists (with experience of CT diagnosis of 6 and 10 years, respectively) who knew nothing about the histopathological condition of each patient were appointed to reevaluate the LN status and CT findings (for example, size, periphery, pancreatic duct dilatation, parenchymal atrophy, and vascular invasion). With reference to the relevant literature and clinical diagnostic experience, the CT diagnostic criteria for metastasis to the lymph nodes in PDAC patients were as follows: the peripancreatic and retroperitoneal lymph nodes with short axis diameter > 10 mm, uneven density, uneven enhancement, internal necrosis, blurred edge, and involvement of surrounding organs or vessels (13, 24). If different opinions from the two radiologists were received for the same patient, an independent expert with 22 years of experience in radiological diagnosis was invited to participate in the discussion to decide the final result. A flow diagram of the whole study is depicted in Figure 1.

FIGURE 1

Figure 1. Radiomics workflow.

Image Acquisition

CT images were acquired from all patients using a 256-slice CT scanner (Brilliance iCT, Philips Medical Systems, Netherlands). The CT scan settings were set as 120 kV; 150–200 mA s; rotation time, 0.75 s; detector collimation, 128 × 0.625 mm; field of view, 350 × 350 mm; matrix, 512 × 512; and slice thickness, 1.5 mm. An anionic contrast medium was injected using an automatic injector at a dose of 1.5 ml/kg at 3.0 ml/s. Arterial phase images were obtained 30 s after contrast medium injection, while venous phase (VP) scans were obtained 45 s after the arterial phase acquisition. All images were uploaded to the picture archiving and communication system (PACS) for further examination.

Tumor Segmentation and Extraction of Radiomics Features

Feature extraction was carried out on 1.5-mm VP CT images because of their better tumor background contrast (25). The window width and window level applied to the tumor segmentation process were 300 and 40 HU, respectively. One radiologist (HF), with 10 years of experience, manually segmented the tumor on each slice around its edge using open-source image processing software (3D Slicer version 4.11.0; Boston, MA, United States). A total of 396 radiomics features was extracted by the software Artificial Intelligence Kit (GE Healthcare, China). The extracted radiomics features were classified into six categories: Histogram features (n = 42), form factor features (n = 9), gray level co-occurrence matrix (GLCM) features (n = 144), run length matrix (RLM) features (n = 180), gray level size zone matrix (GLSZM) features (n = 11), and Haralick features (n = 10). A detailed description of these features can be seen in Supplementary Material I. We calculated all the features in the segmented tumor region within a three-dimensional volume.

To evaluate the reproducibility and accuracy of the features, two radiologists (HF and GJH) reassessed the tumor segmentation of 60 randomly selected patients after 20 days. The two radiologists were both blinded to the clinical diagnosis and pathological condition of each patient. The inter- and intraclass correlation coefficients (ICCs) were taken as measures of good reproducibility. The threshold of the ICC value for a feature with outstanding reproducibility was deemed to be above 0.75 (26).

Feature Selection and Signature Construction

For dimensionality reduction and to avoid overfitting, we designed a three-step procedure to select the optimal features. First, we used both intra- and inter-ICC values > 0.75 as a threshold standard to select the stable radiomics features for the next step. Second, an maximum relevance–minimum redundancy (mRMR) method was selected to eliminate the redundant and irrelevant features, such that 30 features were retained for subsequent selection. Finally, we applied the least absolute shrinkage and selection operator (LASSO) regression algorithm to choose the most reproducible and active characteristics from the remaining 30 features. Those features with non-zero coefficients after the cross-validation penalty procedure in the LASSO regression were assigned to construct the Radiomics score (Rad score) in the training cohort, through a linear combination of their weighted coefficients. The relationship between the Rad score and actual LN status was evaluated in both the training cohort and validation cohort by using a Mann–Whitney U test. We also used receiver operating characteristic (ROC) testing and area under the curve (AUC) analysis to estimate whether the Rad score could correctly distinguish the actual LN status for PDAC patients in both of the two cohorts.

Model Building and Nomogram Development

Univariate analyses were performed on all the clinical and conventional imaging features in the training cohort (including age, gender, CA-199 level, tumor size, tumor location, periphery, CT-reported LN status, CT-reported pancreatic atrophy, CT-reported vascular invasion, and CT-reported pancreatic duct dilatation). A multivariable logistic regression with backward stepwise selection was then conducted by using the variables with P < 0.1 in the univariable regression. Using the likelihood ratio test with Akaike’s information criterion as the stopping rule, a clinical model was built from those variables with P < 0.1 in the multivariate analysis (27, 28). Finally, we constructed a combined multivariable logistic model with Rad scores and the most significant features in the clinical model. To further avoid collinearity, we implemented collinearity diagnosis by checking the variance inflation factor (VIF) for all the factors in the combined model. Those factors with VIF > 5 were excluded from the final model. In order to develop a more understandable evaluation method, we generated a nomogram on the strength of the combined model constructed from the training cohort. Nomogram scores are capable of quantifying the risk of LNM objectively, which can aid in clinical decision-making.

Model Validation

We compared the discriminatory performance of the established models with the ROC curves and AUC values. Thereafter, we used the calibration curves and Hosmer–Lemeshow test to assess the calibration of the nomogram. The above performance of the model was also verified in the validation cohort. We also performed a stratified analysis of the nomogram to test its evaluation efficiency for different human characteristics (n = 172). The adequacy of the performance of the nomogram was assessed by measuring the ROC curves and AUC values in the subgroups including age [≤60 years (young) or >60 years (older)], gender (male or female), and CT-reported LN status (positive or negative).

Clinical Use

For the purpose of determining the value of our nomogram for clinical applications, we adopted decision curve analysis (DCA) to further compare the net benefit obtained by the deployment of the nomogram and the clinical model. The performance of these two models was evaluated at different threshold probabilities, and the model that possessed larger regions under the curves was selected for the better clinical outcome (29).

Statistical Analysis

The Student’s t test was adopted to compare normally distributed variables. Continuous variables that were not normally distributed were analyzed using the Mann–Whitney U test. The discrete variables were compared with the chi-square test. All the statistical analyses that we used in this study were run on R software (version 3.6.2). A detailed description of the R packages that we adopted is provided in Supplementary Material II. A two-tailed P < 0.05 was deemed as possessing statistical significance.

Results

Patients’ Characteristics

Table 1 summarizes the baseline information of all the patients in this study. There were no significant differences between any of the clinical features of the training and the validation groups, neither for patients with or for those without LN metastasis. Thus, there was a good degree of equivalence between the two groups. Only CT-reported LN status and CT-reported parenchymal atrophy showed a significant difference (P < 0.05) between the LNM (+) and LNM (−) group in both the training and the validation cohort.

TABLE 1

Table 1. Characteristics of patients in the training and validation cohorts.

Feature Selection and Radiomics Signature Construction

A total of 396 features were extracted from axial VP CE-CT scans, of which 335 (81.9%) radiomics features were retained after the ICC assessment. Of these, 30 features were retained through the mRMR algorithm for the subsequent LASSO analysis. The LASSO regression was conducted to select the optimized features to construct the final model. Thus, finally, 10 radiomics features were chosen to build the radiomics signature. A detailed description of the selected features can be seen in Figure 2 and Supplementary Table S1. A multilogistic regression-based radiomics signature was constructed using these 10 features, which are represented by the quantitative index designated the Rad score. The formula for calculating the Rad score is presented in Supplementary Material III.

FIGURE 2

Figure 2. Radiomic features selected for signature building.

Evaluation of the Performance of the Rad Score

A significant difference can be seen in the Rad score between patients with or without LN metastasis in the training cohort (P < 0.01), which is confirmed in the validation cohort (P < 0.01; Figure 3A). The Rad score presented an AUC value of 0.90 [95% confidence interval (CI), 0.85, 0.96] in the training cohort and 0.89 (95% CI, 0.80, 0.97) in the validation cohort, documenting very good discriminatory abilities (Figure 3B).

FIGURE 3

Figure 3. The receiver operating characteristic (ROC) curves of the Rad score in the (A) training cohort and the (B) validation cohort. The box-dot plots of the Rad scores in the (C) training cohort and the (D) validation cohort. The orange markers indicate patients with LNM; the green markers indicate patients with non-LNM. The black horizontal line presents the threshold. Patients with Rad scores higher than −0.2635 are classified as LNM; patients with scores lower than −0.2635 are classified as non-LNM.

Nomogram Development and Performance Validation

A detailed description of the multivariable regression analysis can be seen in Table 2. Rad scores, CT-reported LN status, and CT-reported parenchymal atrophy were all significantly correlated with LNM. We constructed a combined model that incorporated Rad scores and the two conventional imaging features and established a nomogram based on this combined model (Figure 4A). In the ROC test, the nomogram displayed a superb ability for evaluating LNM in PDAC patients, with AUCs of 0.92 (95% CI, 0.88–0.97) and 0.95 (95% CI, 0.90–1.00) in the training and validation cohorts, respectively (Figures 4B,C and Table 3). The application of Delong’s test showed that significant differences are present in the AUC values between the combined nomogram and the clinical model (P < 0.001), which confirm its satisfactory predictive performance.

TABLE 2

Table 2. Risk factors for lymph node metastasis in PDAC.

FIGURE 4

Figure 4. (A) The nomogram, combining Rad score, CT-reported parenchymal atrophy, and CT-reported LN status. Receiver operating characteristic (ROC) curves for the nomogram, Rad score, and clinical model in the (B) training and (C) validation cohorts.

TABLE 3

Table 3. Diagnostic performance of models in the training and validation cohorts.

The calibration curves of the nomogram presented a good consistency between predicted and observed LN status in both training and validation cohorts (Figures 5A,B). The Hosmer–Lemeshow test yielded non-significant P values for differences between the two datasets (training cohort, P = 0.31; validation cohort, P = 0.68), documenting that the goodness of fit of our nomogram was acceptable.

FIGURE 5

Figure 5. The calibration curves presented good consistency between the nomogram-predicted lymph node (LN) status and observed LN status in the (A) training cohort and (B) validation cohort.

Stratification analysis revealed that the nomogram had a good capacity to identify lymph nodes in different stratification contexts (Table 4). The AUC value for the combined nomogram was 0.88 (95% CI, 0.82, and 0.94) in the CT-reported LN-negative subgroup, demonstrating its improved recognition capability compared with the traditional imaging methods.

TABLE 4

Table 4. The area under the curve (AUC) values of combined model for stratified analysis in different subgroup.

Clinical Use

Figure 6 presents a DCA using our nomogram. It can be concluded from inspecting the curve that when the threshold probability is over 10% approximately, the nomogram would provide extra diagnostic efficacy over and above the “treat all” or “treat none” scheme of the clinical model.

FIGURE 6

Figure 6. Decision curve analysis for the combined model (nomograms) compared with clinical model in the validation cohort. The decision curve analysis demonstrated that when the threshold probability is over 10% approximately, the nomogram would provide extra diagnostic efficacy over the “treat-all” or “treat-none” scheme and the clinical model.

Discussion

In the present study, we constructed and validated a CT-based radiomics nomogram consisting of the Rad score together with clinical features, which can be used for predicting LN status in PDAC patients. In the ROC test, the combined model and its nomogram exhibited excellent resolution capability in both the training and validation cohorts. The AUC values of the combined model (0.92) and radiomics model (0.90) were both significantly higher than that of the clinical model (0.74). The DCA test showed that the nomogram could effectively facilitate clinical decision-making as well. Considering that only a minority of patients with PDAC can potentially undergo radical resection, an accurate choice of clinical treatment is crucial for PDAC patients. The prognostic value of LN status in PDAC patients has been demonstrated by many investigations, and it is known that even the number of positive LNs also affects therapeutic efficacy (30–32).

Current surgical decision-making depends heavily on imaging diagnosis, despite the fact that the performance of the imaging methodology is not yet completely satisfactory. Although the macroscopic features we observed in the images do have prognostic value for PDAC patients (33), they are not sufficient when it comes to the assessment of LN status. CT is the most often preferred method for preoperative cancer evaluation. In many studies, LNs bigger than 10 mm have been classified as positive LNs (34–36). Nevertheless, the diagnostic efficiency of CT yields only low accuracy and sensitivity. EUS-FNA seems to be a superior solution currently for determining LN metastasis and can be used to extract a piece of tissue and obtain the pathological information on a specific lymph node (37). However, it remains a challenge in clinical practice routinely using endoscopic ultrasonography to access this type of biopsy. Many factors also affect the accuracy of EUS examination, quite randomly (38, 39), thus reducing its value for clinical LN status prediction.

Radiomics detects the heterogeneity of the tumor through the spatial distribution of voxel intensities, acquiring in-depth information from images of the lesion. We constructed a radiomics nomogram to predict LN metastasis by combining the radiomics features and the most relevant clinical characteristics. To facilitate the clinical use of a radiomics model, we constructed a nomogram to visualize and quantize the results of the complex radiomics analysis. Considering the weaknesses of the preceding radiomics models and the doubts about their reproducibility and robustness (40), we took effective measures to guarantee the objectivity and reproducibility of our radiomics model. Changes in tissues that were less correlated with tumor heterogeneity (such as cystic changes) were excluded from the ROI. We preprocessed all the radiomics features to avoid the effects of scale differences. Two radiologists carried out the tumor segmentation step, and ICC coefficients were used to minimize subjectivity and operator error. Both the segmentation and feature extraction software that we used were commonly adopted in earlier investigations and had been verified by those studies (41–43). A three-step approach was devised to reduce the number of features, prevent over fitting, and minimize the collinear features. With all the above measures, a relatively evidence-based and independent radiomics model was constructed for the evaluation of LN status in PDAC patients.

In our radiomics model, we extracted 10 features that could better reflect intratumor heterogeneity and subtle changes in the lesions. The CT-reported LN status and CT-reported parenchymal atrophy also served as independent predictors in the combined model. Previous studies had demonstrated that the CT-reported LN status was significantly related to the pathological LN status in other malignant tumors (44–46), and our study also supports this notion. Fibrosis and parenchymal atrophy is consistently found in PDAC (47), and the tumor microenvironment is likely to be influenced by the reciprocal interactions among fibroblasts and tumor cells in the fibrotic lesions. The degree of pancreatic atrophy is directly related to the malignancy of the tumor and reflects the severity of tissue fibrosis (48). Although to the best of our knowledge, there are no published studies indicating that pancreatic atrophy is an independent factor for LN status in PDAC, we have reason to believe that it does have a potential association with LNM in such patients. Compared with the previous studies using radiomics to evaluate the LN status for PDAC patients (49, 50), we believe that our approach offers advantages for the following reasons: (1) We performed a stratified analysis to further evaluate the prediction efficiency of our model, which can determine the clinical application potential of our model under different conditions. (2) In addition to the CT-reported LN status, we adopted more conventional CT imaging signs in the clinical model. The combination of radiomics and traditional imaging signs may improve the clinical acceptability of radiomics.

Our study has several limitations as follows: (1) One of the main drawbacks of radiomics research is that the poor interpretation of the radiomics features has always hindered the clinical promotion of radiomics. Although many studies tried to generate the correlations from the perspective of grayscale intensity and matrix uniformity, it is still difficult to directly connect the radiomics features with the clinical status. This problem also existed in our study. (2) The patients in our study were all recruited from one hospital. Further external validation with considerably larger data sets should be carried out to testify to the robustness and prediction accuracy of the model. (3) The study only concentrated on the occurrence or lack of LN metastasis in PDAC patients. The number of different metastatic LNs is also important according to the latest cancer staging guidelines (51). The predictive accuracy of radiomics for specific N stage (N1–N2) needs further investigation. (4) Other clinical and imaging features may also be valuable for the construction of the predictive model, but we excluded them from the present study for reasons of data integrity and only selected the most reasonable features.

In conclusion, we have established and verified a novel radiomics nomogram to evaluate LN status in PDAC patients. The model consisted of Rad scores, CT-reported LN status, and CT-reported parenchymal atrophy. Our results demonstrate that the nomogram could likely be conducive to enhancing an accurate auxiliary diagnosis and increasing the optimization of appropriate clinical treatment.

Data Availability Statement

The datasets generated for this study are available on request to the corresponding author.

Ethics Statement

The studies involving human participants were reviewed and approved by the Institutional Review Board of Huashan Hospital. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

Author Contributions

JG and FH contributed equally to the study and designed and carried out the experiments. YJ and XW collected and sorted the data. JZ supervised and revised the manuscript. All authors contributed to the article and approved the submitted version.

Funding

The study was funded by the National Key Research and Development Project in China (No. 2017YFC0113405).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fonc.2020.01654/full#supplementary-material

References

1. Siegel, RL, Miller, KD, and Jemal, A. Cancer statistics, 2020. CA Cancer J Clin. (2020) 70:7–30. doi: 10.3322/caac.21590

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Zhan, H, Xu, J, Wang, L, Zhang, G, and Hu, S. Lymph node ratio is an independent prognostic factor for patients after resectionof pancreatic cancer. World J Surg Oncol. (2015) 13:105. doi: 10.1186/s12957-015-0510-0

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Winter, JM, Cameron, JL, Campbell, KA, Arnold, MA, Chang, DC, Coleman, J, et al. 1423 pancreaticoduodenectomies for pancreatic cancer: a single-institutionexperience. J Gastrointest Surgery. (2006) 10:1210–1. doi: 10.1016/j.gassur.2006.08.018

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Burke, EE, Marmor, S, Virnig, BA, Tuttle, TM, and Jensen, EH. Lymph node evaluation for pancreatic adenocarcinoma and its value as a qualitymetric. J Gastrointest Surgery. (2015) 19:2162–70. doi: 10.1007/s11605-015-2969-y

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Zeng, H, Chen, W, Zheng, R, Zhang, S, Ji, JS, Zou, X, et al. Changing cancer survival in China during 2003-15: a pooled analysis of 17population-based cancer registries. Lancet Global Health. (2018) 6:e555–67. doi: 10.1016/S2214-109X(18)30127-X

CrossRef Full Text | Google Scholar

6. Ashfaq, A, Pockaj, BA, Gray, RJ, Halfdanarson, TR, and Wasif, N. Nodal counts and lymph node ratio impact survival after distal pancreatectomy forpancreatic adenocarcinoma. J Gastrointest Surgery. (2014) 18:1929–35. doi: 10.1007/s11605-014-2566-5

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Strobel, O, Hinz, U, Gluth, A, Hank, T, Hackert, T, Bergmann, F, et al. Pancreatic adenocarcinoma: number of positive nodes allows to distinguish severalN categories. Ann Surg. (2015) 261:961–9. doi: 10.1097/SLA.0000000000000814

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Takhar, AS, Palaniappan, P, Dhingsa, R, and Lobo, DN. Recent developments in diagnosis of pancreatic cancer. BMJ (Clin Res Ed). (2004) 329:668–73. doi: 10.1136/bmj.329.7467.668

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Tamm, EP, Balachandran, A, Bhosale, PR, Katz, MH, Fleming, JB, Lee, JH, et al. Imaging of pancreatic adenocarcinoma: update on staging/resectability. Radiol Clin N Am. (2012) 50:407–28. doi: 10.1016/j.rcl.2012.03.008

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Dai, C, Yang, Z, Xue, L, and Li, Y. Application value of multi-slice spiral computed tomography for imagingdetermination of metastatic lymph nodes of gastric cancer. World J Gastroentero. (2013) 19:5732–7. doi: 10.3748/wjg.v19.i34.5732

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Saito, T, Kurokawa, Y, Takiguchi, S, Miyazaki, Y, Takahashi, T, Yamasaki, M, et al. Accuracy of multidetector-row CT in diagnosing lymph node metastasis in patients with gastric cancer. Eur Radiol. (2015) 25:368–74. doi: 10.1007/s00330-014-3373-9

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Parry, K, Haverkamp, L, Bruijnen, RCG, Siersema, PD, Offerhaus, GJA, Ruurda, JP, et al. Staging of adenocarcinoma of the gastroesophageal junction. Eur J Surg Oncol. (2016) 42:400–6. doi: 10.1016/j.ejso.2015.11.014

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Roche, CJ, Hughes, ML, Garvey, CJ, Campbell, F, White, DA, Jones, L, et al. CT and pathologic assessment of prospective nodal staging in patients with ductaladenocarcinoma of the head of the pancreas. AJR Am J Roentgenol. (2003) 180:475–80. doi: 10.2214/ajr.180.2.1800475

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Kauhanen, SP, Komar, G, Seppanen, MP, Dean, KI, Minn, HR, Kajander, SA, et al. A prospective diagnostic accuracy study of 18F-fluorodeoxyglucose positronemission tomography/computed tomography, multidetector row computed tomography, and magnetic resonance imaging in primary diagnosis and staging of pancreatic cancer. Ann Surg. (2009) 250:957–63. doi: 10.1097/SLA.0b013e3181b2fafa

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Savides, TJ, Donohue, M, Hunt, G, Al-Haddad, M, Aslanian, H, Ben-Menachem, T, et al. EUS-guided FNA diagnostic yield of malignancy in solid pancreatic masses: abenchmark for quality performance measurement. Gastrointest Endosc. (2007) 66:277–82.

Google Scholar

16. Mitsuhashi, T, Ghafari, S, Chang, CY, and Gu, M. Endoscopic ultrasound-guided fine needle aspiration of the pancreas:cytomorphological evaluation with emphasis on adequacy assessment, diagnosticcriteria and contamination from the gastrointestinal tract. Cytopathology. (2006):34–41. doi: 10.1111/j.1365-2303.2006.00277.x

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Chhieng, DC, Jhala, D, Jhala, N, Eltoum, I, Chen, VK, Vickers, S, et al. Endoscopic ultrasound-guided fine-needle aspiration biopsy: a study of 103 cases. Cancer Am Cancer Soc. (2002) 96:232–9. doi: 10.1002/cncr.10714

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Eloubeidi, MA, Jhala, D, Chhieng, DC, Chen, VK, Eltoum, I, Vickers, S, et al. Yield of endoscopic ultrasound-guided fine-needle aspiration biopsy in patientswith suspected pancreatic carcinoma. Cancer. (2003) 99:285–92. doi: 10.1002/cncr.11643

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Sakamoto, H, Kitano, M, Komaki, T, Noda, K, Chikugo, T, Dote, K, et al. Prospective comparative study of the EUS guided 25-gauge FNA needle with the19-gauge Trucut needle and 22-gauge FNA needle in patients with solid pancreatic masses. J Gastroenterol Hepatol. (2009):384–90. doi: 10.1111/j.1440-1746.2008.05636.x

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Iglesias-Garcia, J, Dominguez-Munoz, JE, Abdulkader, I, Larino-Noia, J, Eugenyeva, E, Lozano-Leon, A, et al. Influence of on-site cytopathology evaluation on the diagnostic accuracy ofendoscopic ultrasound-guided fine needle aspiration (EUS-FNA) of solid pancreaticmasses. Am J Gastroenterol. (2011) 106:1705–10. doi: 10.1038/ajg.2011.119

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Wang, Y, Liu, W, Yu, Y, Liu, J, Xue, H, Qi, Y, et al. CT radiomics nomogram for the preoperative prediction of lymph node metastasis ingastric cancer. Eur Radiol. (2020) 30:976–86. doi: 10.1007/s00330-019-06398-z

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Xu, L, Yang, P, Liang, W, Liu, W, Wang, W, Luo, C, et al. A radiomics approach based on support vector machine using MR images forpreoperative lymph node status evaluation in intrahepatic cholangiocarcinoma. Theranostics. (2019) 9:5374–85. doi: 10.7150/thno.34149

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Huang, Y, Liang, C, He, L, Tian, J, Liang, C, Chen, X, et al. Development and validation of a radiomics nomogram for preoperative prediction oflymph node metastasis in colorectal cancer. J Clin Oncol. (2016) 34:2157–64. doi: 10.1200/JCO.2015.65.9128

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Chun, YS, Pawlik, TM, and Vauthey, J. 8th edition of the AJCC Cancer staging manual: pancreas and hepatobiliary cancers. Ann Surg Oncol. (2018) 25:845–7. doi: 10.1245/s10434-017-6025-x

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Permuth, JB, Choi, J, Balarunathan, Y, Kim, J, Chen, D, Chen, L, et al. Combining radiomic features with a miRNA classifier may improve prediction of malignant pathology for pancreatic intraductal papillary mucinous neoplasms. Oncotarget. (2016) 7:85785–97. doi: 10.18632/oncotarget.11768

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Barry, WT, Kernagis, DN, Dressman, HK, Griffis, RJ, Hunter, JD, Olson, JA, et al. Intratumor heterogeneity and precision of microarray-based predictors of breastcancer biology and clinical outcome. J Clin Oncol. (2010) 28:2198–206. doi: 10.1200/JCO.2009.26.7245

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Sauerbrei, W, Royston, P, and Binder, H. Selection of important variables and determination of functional form forcontinuous predictors in multivariable model building. Stat Med. (2007) 26:5512–28. doi: 10.1002/sim.3148

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Collins, GS, Reitsma, JB, Altman, DG, and Moons, KGM. Transparent reporting of a multivariable prediction model for individualprognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ. (2015) 350:55–63. doi: 10.7326/M14-0697

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Van Calster, B, Wynants, L, Verbeek, JFM, Verbakel, JY, Christodoulou, E, Vickers, AJ, et al. Reporting and interpreting decision curve analysis: a guide for investigators. Eur Urol. (2018) 74:796–804. doi: 10.1016/j.eururo.2018.08.038

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Murakami, Y, Uemura, K, Sudo, T, Hayashidani, Y, Hashimoto, Y, Nakashima, A, et al. Number of metastatic lymph nodes, but not lymph node ratio, is an independentprognostic factor after resection of pancreatic carcinoma. J Am Coll Surg. (2010):196–204. doi: 10.1016/j.jamcollsurg.2010.03.037

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Pawlik, TM, Gleisner, AL, Cameron, JL, Winter, JM, Assumpcao, L, Lillemoe, KD, et al. Prognostic relevance of lymph node ratio following pancreaticoduodenectomy forpancreatic cancer. Surgery. (2007) 141:610–8. doi: 10.1016/j.surg.2006.12.013

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Tol, JAMG, Brosens, LAA, van Dieren, S, van Gulik, TM, Busch, ORC, Besselink, MGH, et al. Impact of lymph node ratio on survival in patients with pancreatic andperiampullary cancer. Br J Surgery. (2015) 102:237–45. doi: 10.1002/bjs.9709

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Hori, S, Shimada, K, Ino, Y, Oguro, S, Esaki, M, Nara, S, et al. Macroscopic features predict outcome in patients with pancreatic ductal adenocarcinoma. Virchows Arch. (2016) 469:621–34. doi: 10.1007/s00428-016-2026-6

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Howard, TJ, Chin, AC, Streib, EW, Kopecky, KK, and Wiebke, EA. Value of helical computed tomography, angiography, and endoscopic ultrasound indetermining resectability of periampullary carcinoma. Am J Surg. (1997) 174:237–41. doi: 10.1016/S0002-9610(97)00132-3

CrossRef Full Text | Google Scholar

35. Imai, H, Doi, R, Kanazawa, H, Kamo, N, Koizumi, M, Masui, T, et al. Preoperative assessment of para-aortic lymph node metastasis in patients withpancreatic cancer. Int J Clin Oncol. (2010) 15:294–300. doi: 10.1007/s10147-010-0066-5

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Nanashima, A, Tobinaga, S, Abo, T, Hatano, K, Takeshita, H, Nonaka, T, et al. Evaluation of surgical resection for pancreatic carcinoma at a Japanese singlecancer institute. Hepato Gastroenterol. (2012) 59:911–5.

Google Scholar

37. Matsubayashi, H, Matsui, T, Yabuuchi, Y, Imai, K, Tanaka, M, Kakushima, N, et al. Endoscopic ultrasonography guided-fine needle aspiration for the diagnosis ofsolid pancreaticobiliary lesions: clinical aspects to improve the diagnosis. World J Gastroentero. (2016) 22:628–40. doi: 10.3748/wjg.v22.i2.628

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Yoo, J, Kistler, CA, Yan, L, Dargan, A, and Siddiqui, AA. Endoscopic ultrasound in pancreatic cancer: innovative applications beyond thebasics. J Gastrointest Oncol. (2016) 7:1019–29. doi: 10.21037/jgo.2016.08.07

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Lee, YN, Moon, JH, Kim, HK, Choi, HJ, Lee, SH, Choi, MH, et al. A triple approach for diagnostic assessment of endoscopic ultrasound-guided fine needle aspiration in pancreatic solid masses and lymph nodes. Digest Dis Sci. (2014) 59:2286–93. doi: 10.1007/s10620-014-3119-1

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Welch, ML, McIntosh, C, Haibe-Kains, B, Milosevic, MF, Wee, L, Dekker, A, et al. Vulnerabilities of radiomic signature development: the need for safeguards. Radiother Oncol. (2019) 130:2–9. doi: 10.1016/j.radonc.2018.10.027

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Gao, C, Xiang, P, Ye, J, Pang, P, Wang, S, and Xu, M. Can texture features improve the differentiation of infiltrative lungadenocarcinoma appearing as ground glass nodules in contrast-enhanced CT? Eur J Radiol. (2019) 117:126–31. doi: 10.1016/j.ejrad.2019.06.010

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Shu, Z, Fang, S, Ding, Z, Mao, D, Cai, R, Chen, Y, et al. MRI-based Radiomics nomogram to detect primary rectal cancer with synchronousliver metastases. Sci Rep UK. (2019) 9:3374. doi: 10.1038/s41598-019-39651-y

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Shu, Z, Fang, S, Ye, Q, Mao, D, Cao, H, Pang, P, et al. Prediction of efficacy of neoadjuvant chemoradiotherapy for rectal cancer: the value of texture analysis of magnetic resonance images. Abdominal Radiol (New York). (2019) 44:3775–84. doi: 10.1007/s00261-019-01971-y

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Han, L, Zhu, Y, Liu, Z, Yu, T, He, C, Jiang, W, et al. Radiomic nomogram for prediction of axillary lymph node metastasis in breastcancer. Eur Radiol. (2019) 29:3820–9. doi: 10.1007/s00330-018-5981-2

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Cong, M, Feng, H, Ren, J, Xu, Q, Cong, L, Hou, Z, et al. Development of a predictive radiomics model for lymph node metastases inpre-surgical CT-based stage IA non-small cell lung cancer. Lung cancer (Amsterdam Netherlands). (2020) 139:73–9. doi: 10.1016/j.lungcan.2019.11.003

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Wang, F, Zhang, B, Wu, X, Liu, L, Fang, J, Chen, Q, et al. Radiomic nomogram improves preoperative T category accuracy in locally advancedlaryngeal carcinoma. Front Oncol. (2019) 9:1064. doi: 10.3389/fonc.2019.01064

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Aimoto, T, Uchida, E, Nakamura, Y, Matsushita, A, Katsuno, A, Chou, K, et al. Multicentric pancreatic intraepithelial neoplasias (PanINs) presenting with theclinical features of chronic pancreatitis. J Hepato Biliary Pancreatic Surgery. (2008) 15:549–53. doi: 10.1007/s00534-007-1269-7

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Fukumoto, T, Watanabe, T, Hirai, I, and Kimura, W. Pancreatic volume is one of the independent prognostic factors for resectablepancreatic ductal adenocarcinomas. J Hepato Biliary Pancreat Sci. (2016) 23:472–9. doi: 10.1002/jhbp.365

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Bian, Y, Guo, S, Jiang, H, Gao, S, Shao, C, Cao, K, et al. Relationship between radiomics and risk of lymph node metastasis in pancreaticductal adenocarcinoma. Pancreas. (2019) 48:1195–203. doi: 10.1097/MPA.0000000000001404

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Li, K, Yao, Q, Xiao, J, Li, M, Yang, J, Hou, W, et al. Contrast-enhanced CT radiomics for predicting lymph node metastasis in pancreatic ductal adenocarcinoma: a pilot study. Cancer Imaging. (2020) 20:12. doi: 10.1186/s40644-020-0288-3

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Van Roessel, S, Kasumova, GG, Verheij, J, Najarian, RM, Maggino, L, de Pastena, M, et al. International validation of the eighth edition of the american joint committee on cancer (AJCC) TNM Staging system in patients with resected pancreatic cancer. Jama Surg. (2018) 153:e183617. doi: 10.1001/jamasurg.2018.3617

PubMed Abstract | CrossRef Full Text | Google Scholar