Construction and validation of an endoscopic ultrasonography-based ultrasomics nomogram for differentiating pancreatic neuroendocrine tumors from pancreatic cancer

Objectives: To develop and validate various ultrasomics models based on endoscopic ultrasonography (EUS) for retrospective differentiating pancreatic neuroendocrine tumors (PNET) from pancreatic cancer.

Methods: A total of 231 patients, comprising 127 with pancreatic cancer and 104 with PNET, were retrospectively enrolled. These patients were randomly divided into either a training or test cohort at a ratio of 7:3. Ultrasomics features were extracted from conventional EUS images, focusing on delineating the region of interest (ROI) for pancreatic lesions. Subsequently, dimensionality reduction of the ultrasomics features was performed by applying the Mann-Whitney test and least absolute shrinkage and selection operator (LASSO) algorithm. Eight machine learning algorithms, namely logistic regression (LR), light gradient boosting machine (LightGBM), multilayer perceptron (MLP), random forest (RF), extra trees, k nearest neighbors (KNN), support vector machine (SVM), and extreme gradient boosting (XGBoost), were employed to train prediction models using nonzero coefficient features. The optimal ultrasomics model was determined using a ROC curve and utilized for subsequent analysis. Clinical-ultrasonic features were assessed using both univariate and multivariate logistic regression. An ultrasomics nomogram model, integrating both ultrasomics and clinical-ultrasonic features, was developed.

Results: A total of 107 EUS-based ultrasomics features were extracted, and 6 features with nonzero coefficients were ultimately retained. Among the eight ultrasomics models based on machine learning algorithms, the RF model exhibited superior performance with an AUC= 0.999 (95% CI 0.9977 - 1.0000) in the training cohort and an AUC= 0.649 (95% CI 0.5215 - 0.7760) in the test cohort. A clinical-ultrasonic model was established and evaluated, yielding an AUC of 0.999 (95% CI 0.9961 - 1.0000) in the training cohort and 0.847 (95% CI 0.7543 - 0.9391) in the test cohort. Subsequently, the ultrasomics nomogram demonstrated a significant improvement in prediction accuracy in the test cohort, as evidenced by an AUC of 0.884 (95% CI 0.8047 - 0.9635) and confirmed by the Delong test. The calibration curve and decision curve analysis (DCA) depicted this ultrasomics nomogram demonstrated superior accuracy. They also yielded the highest net benefit for clinical decision-making compared to alternative models.

Conclusions: A novel ultrasomics nomogram was proposed and validated, that integrated clinical-ultrasonic and ultrasomics features obtained through EUS, aiming to accurately and efficiently identify pancreatic cancer and PNET.

Introduction

Pancreatic neuroendocrine tumors (PNET) are a rare occurrence, constituting approximately 3%-5% of all pancreatic tumors, and exhibit a significant degree of heterogeneity (1). These tumors originate from pancreatic endocrine tissues and rank as the second most prevalent pancreatic tumor, surpassed only by pancreatic cancer (2). PNET are characterized by extremely variable biological features spanning from low-grade malignant tumors to immensely aggressive ones (3). Based on the capability to secrete biologically active hormones and characteristic clinical symptoms, PNET can be categorized into two distinct categories: namely functional (F-PNET) and non-functional (NF-PNET) (4). NF-PNET exhibit a greater incidence and a more unfavorable prognosis than F-PNET, except the insulinomas (5). The preoperative identification of PNET poses a considerable challenge, relying primarily on pathological examination and immunohistochemistry, with pancreatic cancer being the most critical differential diagnosis (6). Consequently, the prompt and accurate diagnosis and treatment of PNET hold utmost significance (7).

Currently, a range of diagnostic imaging modalities, such as computed tomography (CT), magnetic resonance imaging (MRI), and transabdominal ultrasonography (US) are commonly applied for the diagnosis of PNET. Compared to CT and MRI, endoscopic ultrasonography (EUS) is regarded as one of the most accurate imaging modalities for the diagnosis of pancreatic diseases because of its ability to provide high-definition images of the pancreas and its sensitivity ranging from 57% to 94% (8). According to the consensus guidelines of the European Neuroendocrine Tumor Society (ENETS) in 2023, EUS is considered the preferred imaging modality following negative findings from alternative noninvasive imaging techniques. This preference is due to EUS’s ability to offer meticulous observation and estimation of PNET, as well as its capacity to conduct a comprehensive scan of the whole pancreas (9).

Currently, ongoing advancements in computer-aided detection (CAD) and artificial intelligence (AI) have contributed to the gradual emergence of radiomics as a promising research domain. Radiomics enables the extraction and analysis of numerous objective and internal image features through high-throughput techniques. These features are subsequently utilized to develop diverse tumor diagnosis and prediction models using various machine learning, deep learning, and other algorithmic approaches (10). Numerous studies have elucidated the utility of CT-based and MRI-based radiomics in diagnosing and predicting PNET, showcasing its efficacy (11, 12). However, MRI is contraindicated for certain populations, including individuals with claustrophobia or metal implants. Additionally, the time-consuming and costly nature of MRI restricts its widespread clinical utility (13) Furthermore, CT presents drawbacks such as radiation exposure risks and potential harm from contrast agents (14).

Ultrasomics, a subfield of radiomics, involves the conversion of digitally encrypted medical images containing tumor pathophysiology information into high-dimensional data that can be analyzed. This technique has demonstrated efficacy in the precise diagnosis of a range of malignant tumors, including liver cancer, thyroid cancer, and breast cancer, yielding favorable outcomes (15). However, the efficiency of ultrasomics based on normal EUS images in enhancing the clinical diagnostic efficiency of PNET, despite EUS being acknowledged as an exceptional imaging modality, has not been validated.

In this research, a range of prevalent machine learning algorithms was employed to develop and assess an effective combined nomogram model based on ultrasomics and clinical signatures, to distinguish PNET from pancreatic cancer.

Materials and methods

Patient population

This retrospective study at our institution was approved by the institutional ethics review board, which waived the requirement for patient approval or signed informed consent to review medical images or information. A total of 231 patients underwent pancreatic surgery operation or EUS-guided fine-needle aspiration (EUS-FNA) at our institution (The First Affiliated Hospital of Guangxi Medical University) between January 2013 and October 2023, comprising 127 with pancreatic cancer and 104 with PNET, were retrospectively enrolled. The inclusion and exclusion criteria are outlined as follows.

The inclusion criteria for patients were as follows: (1) underwent preoperative EUS scan of the pancreas meticulously; (2) had pancreatic cancers or PNET confirmed by postoperative pathology or EUS-FNA pathology; and (3) had complete and clear EUS images available before the patient’s preoperative or pathological biopsies. (4) Patients who received no chemotherapy or radiotherapy before EUS. The exclusion criteria for patients were as follows:(1) inability to display the whole lesion; (2) significant motion artifacts or noticeable noise; (3) presence of other types of tumors. The enrolled patients were randomized into a training cohort and a test cohort at a ratio of 7:3. The flowchart for enrolling the study population is shown in Figure 1.

Figure 1

Figure 1 Flowchart for enrolling the study population.

EUS image acquisition

The study employed the standard dynamic EUS scan procedure using the EU-ME2 (Olympus, Japan) and SU-9000 (FUJIFILM, Japan) devices. A highly experienced EUS specialist with a record of more than 5000 EUS procedures meticulously scanned the entire pancreatic region and obtained high-resolution images of the pancreatic masses. At the same time, to reduce the image bias caused by different devices as much as possible, these images were consistently standardized with a window width of 250 Hounsfield units (HU) and a window level of 125 HU. The imaging records were gathered by retrieving data from our institutional Picture Archive and Communication System (PACS).

Ultrasonic and clinical data analysis

This study retrospectively analyzed multiple clinical parameters, such as age, gender, and pathological diagnosis. To enhance the study’s credibility, all EUS images were meticulously examined and evaluated by two proficient UES experts, each with 6–7 years of experience in pancreatic EUS. In the case of disagreement, a consensus would be reached by consultation. Crucially, these experts were blinded to the histopathological and clinical data about the analyzed cases.

This study examined various parameters and features of the pancreatic masses, including its location, echo characteristics, uniformity of the echo, maximum diameter, shape, margin characteristics, and the presence of calcifications or cystic degeneration, via EUS. A detailed explanation of the ultrasonic features can be found in the Supplementary Material. In instances where multiple lesions were present in the pancreas, the analysis primarily concentrated on the largest lesion with confirmed pathology.

Image segmentation

The images were stored in Digital Imaging and Communications in Medicine (DICOM) format. Two EUS specialists, each with 6 and 7 years of experience, manually delineated the region of interest (ROI) using a gray-scale EUS image of the most extensive long-axis cross-section layer for pancreatic masses. This segmentation was performed utilizing ITK-SNAP software (version 3.8.1), accessed at http://www.itksnap.org. The specialists were unaware of the patients’ histopathological diagnosis. The lesions were captured carefully along the margins on conventional EUS images, excluding adjacent normal tissue, vessels, bile ducts, and pancreatic ducts. A comprehensive diagram can be found in Figure 2.

Figure 2

Figure 2 The workflow for this study.

Standardization techniques were employed to preprocess the images and data, thereby guaranteeing the replicability of the results. Both intra-observer and inter-observer replicability were assessed using the intra-class correlation coefficient (ICC). A group of 102 patients, comprising 81 individuals with pancreatic cancer and 21 with PNET, was randomly chosen, and following a two-week interval, the same EUS specialists performed ROI segmentation once more. An ICC value exceeding 0.9 indicated a remarkable level of agreement.

Ultrasomics feature extraction

The classification of handcrafted features can be divided into three distinct categories: geometric, intensity, and textural. Geometric features pertain to the three-dimensional morphological attributes of tumors. The intensity features encompass the statistical distribution of voxel intensities within the tumor in the first order. Conversely, textural features describe patterns and higher-order spatial distributions of intensities. This article utilized a range of techniques, including Gray Level Co-occurrence Matrix (GLCM), Gray Level Run Length Matrix (GLRLM), Gray Level Size Zone Matrix (GLSZM), and Neighborhood Gray-level Difference Matrix (NGTDM), to extract texture features. The algorithms employed to extract radiomic features were based on the image biomarker standardization initiative (IBSI) (16).

Ultrasomics feature selection

To ascertain the dependability of the ultrasomics features, we conducted a Mann−Whitney U test to compare the PNET and pancreatic cancer cohorts, followed by feature screening. Subsequently, only ultrasomics features exhibiting p<0.05 were retained for subsequent analysis.

Correlation coefficient screening was conducted utilizing Spearman’s rank correlation coefficient to assess the interrelationship between each ultrasomics feature, aiming to ensure the robustness of the features (Supplementary Figure 1). Any feature with a correlation coefficient greater than 0.9 between any two features was retained with one of them. A greedy recursive deletion method was employed for feature filtration to enhance feature representation, systematically removing the feature with the highest redundancy within the existing set. As a result, a total of 8 ultrasomics features were ultimately preserved.

Finally, the 10-fold cross-validation method was used to identify ultrasomics features with nonzero coefficients using the least absolute shrinkage and selection operator (LASSO) regression model. All ultrasomics feature selection procedures were conducted within the training cohort and subsequently applied to the test cohort. Ultrasomic features with non-zero coefficients were selected for inclusion in the regression model and combined to form an ultrasomics signature. Subsequently, each patient was assigned an ultrasomics score by applying a linear combination of the selected ultrasomics features and their corresponding coefficients. LASSO regression analysis was conducted utilizing the Python scikit-learn package.

Construction of ultrasomics signature model

The ultimate retained ultrasomics features were utilized in the development of ultrasomics models. To identify a classifier model with optimal tumor data recognition, our study employed eight prominent machine learning algorithms training models, including logistic regression (LR), light gradient boosting machine (LightGBM), multilayer perceptron (MLP), random forest (RF), extra trees, k nearest neighbors (KNN), support vector machine (SVM), and extreme gradient boosting (XGBoost). The definitive ultrasomics models were obtained through 5-fold cross-validation. The diagnostic efficacy of these different machine learning models was assessed through the evaluation of metrics such as the area under the receiver operating characteristic curve (AUC), sensitivity, accuracy, specificity, positive predictive value (PPV), and negative predictive value (NPV). Ultimately, the most optimal ultrasomics model was chosen and defined as the ultrasomics signature model.

Clinical- ultrasonic signature model

Furthermore, we conducted univariate logistic regression analysis on each clinical predictor variable, encompassing both clinical and ultrasonic characteristics. To determine statistically significant clinical-ultrasonic features and establish the clinical-ultrasonic signature model, we performed a multivariate logistic regression. This allowed us to calculate the odds ratio (OR) and 95% confidence interval (CI) for each variable.

The identical ultrasomics signature model was performed to construct the clinical-ultrasonic signature model through the same machine learning algorithm. To ensure equitable comparison, a fixed 5-fold cross-validation and test cohort were utilized. The performance of this clinical-ultrasonic signature model was evaluated using various metrics such as the area under the curve (AUC), accuracy, sensitivity, specificity, PPV, and NPV. Additionally, the net benefit of the clinical-ultrasonic signature model in identifying pancreatic cancer and PNET was quantified through the use of decision curve analysis (DCA), which involved the application of diverse threshold probabilities in both the training and test cohorts.

Ultrasomics nomogram establishment and assessment

Finally, an ultrasomics nomogram was developed in the test cohort to intuitively and efficiently evaluate the incremental predictive value of the ultrasomics signature for the clinical-ultrasonic signature. Based on logistic regression analysis, the ultrasomics nomogram was constructed by integrating the ultrasomics signature with the clinical-ultrasonic signature. The consistency between the prediction of the nomogram and the actual observation was compared by calculating the calibration curve, which compared the prediction of the nomogram with the actual observation.

The diagnostic effectiveness of the ultrasomics nomogram was evaluated in both the training and test cohorts by constructing receiver operating characteristic (ROC) curves. Subsequently, the Delong test was conducted to compare the performance of the ultrasomics nomogram, clinical-ultrasonic signature model, and ultrasomics signature model in terms of the AUC.

The calibration efficiency of the ultrasomics nomogram was assessed by constructing calibration curves. At the same time, the Hosmer-Lemeshow (H-L) analytical fit was employed to evaluate the calibration capability of the ultrasomics nomogram. Additionally, mapping DCA was utilized to evaluate the clinical utility of these predictive models.

Statistical analysis

We compared the clinical and ultrasonic features of the participants utilizing an independent sample t-test, Mann-Whitney U test, or X² test, where appropriate. The threshold was set at a two-tailed p-value < 0.05, indicating statistical significance. Table 1 shows the baseline clinical and ultrasonic features of the participants in the training and test cohorts respectively.

Table 1

Table 1 Clinical and ultrasonic characteristics in the training and test cohorts.

Results

Patient population and ultrasonic characteristics

A total of 231 patients (102 women, 129 men) with pancreatic tumors, comprising 127 patients with pancreatic cancer and 104 patients with PNET, were included in this study, including 161 patients in the training cohort and 70 patients in the testing cohort. The findings illustrated that there was no significant difference in the gender ratio between the patients in the training and test cohorts. All the details of the clinical and ultrasonic characteristics are delineated in Table 1. The outcomes of this study revealed significant differences in age, maximum diameter, shape, margin characteristics, echo characteristics, and the presence of cystic degeneration between pancreatic cancer patients and PNET patients in the training cohort. However, no significant differences were observed in terms of echo characteristics or uniformity of the echo in the test cohort. The results of univariate and multivariable logistic regression analyses indicated that age, maximum diameter, and shape independently predicted the presence of PNET.

The findings indicated that elderly individuals (OR 0.989; 95% CI 0.985 to 0.992), those with an enlarged maximum diameter (OR 0.990; 95% CI 0.987 to 0.993), and those with an irregular shape (OR 1.242; 95% CI 1.133 to 1.361) were more likely to be diagnosed with pancreatic cancer (Table 2; Figure 3).

Table 2

Table 2 Univariate and multivariable logistic regression analyses for selecting clinical and ultrasonic features.

Figure 3

Figure 3 The forest map of univariate logistic regression of clinical and ultrasonic features. *** means P < 0.001.

Ultrasomics feature extraction and screening

A comprehensive set of 7 categories and 107 manually derived ultrasomics features were obtained, comprising 18 first-order features, and 14 shape features, while the remaining features were texture features. Exhaustive information regarding these handcrafted features can be found in the Supplementary Materials.

An in-house feature analysis program, PyRadiomics, was utilized to extract all handcrafted features. Supplementary Figure 2 presents the complete set of ultrasomics features along with their corresponding p-value results. Following the downscaling of ultrasomics features and LASSO logistic regression, a total of 6 ultrasomics features with nonzero coefficients were retained. The coefficients and mean standard errors (MSEs) of LASSO regression, resulting from the 10-fold validation, are depicted in Figure 4.

Figure 4

Figure 4 Ultrasomics feature selection with the LASSO regression model. (A) The LASSO model’s tuning parameter (λ) selection used 10-fold cross-validation via minimum criterion. The vertical lines illustrate the optimal value of the LASSO tuning parameter (λ). (B) LASSO coefficient profile plot with different log (λ) was displayed. The vertical dashed lines represent 6 ultrasomics features with nonzero coefficients selected with the optimal λ value.

The coefficients of these retained 6 ultrasomics features are depicted in Figure 5, while the Ultrasomics score is shown as follows:

Ultrasomics score =0.45022 - 0.03094*original_firstorder_Skewness

-0.02680*original_glcm_ClusterShade

-0.05500*original_glrlm_ShortRunEmphasis

-0.05942*original_glszm_ZoneVariance

-0.00677*original_ngtdm_Busyness

-0.01459*original_ngtdm_Strength

Figure 5

Figure 5 The bar graph of 6 ultrasomics features that achieved nonzero coefficients.

Ultrasomics signature and its performance

The ROC curers and AUCs of the eight ultrasomics machine-learning models obtained with the eight mainstream machine-learning algorithms in the training and test cohorts are presented in Figure 6. Additionally, more details are shown in Table 3. Concerning the training cohort, the ExtraTrees model demonstrated superior performance, achieving an AUC of 1.000 and an accuracy of 1.000. Conversely, in the test cohort, the RF model emerged as the most effective ultrasomics model, exhibiting an AUC of 0.649, an accuracy of 0.600, a sensitivity of 0.567, a specificity of 0.775, a PPV of 0.550, and an NPV of 0.620.

Figure 6

Figure 6 The ROC curves of different ultrasomics models based on eight machine-learning algorithms for predicting PNET. (A) The ROC curves of different ultrasomics models in the training cohort. (B) The ROC curves of different ultrasomics models in the test cohort.

Table 3

Table 3 Diagnostic performance of different models for predicting PNET in training and test cohorts.

The ExtraTrees and XGBoost models presented a tendency toward overfitting in the training and test cohort. Contrarily, the RF model seemed to own better consistency between the training and test cohorts, with a great net benefit as presented in the DCA curve (Supplementary Figure 3). To establish the stability and sustainability of the ultrasomics machine-learning model, the RF model was determined to be the most suitable for subsequent analysis and defined as the ultrasomics signature model. Consequently, RF was selected as the foundational algorithm for developing the clinical-ultrasonic signature.

Ultrasomics nomogram construction and assessment

Both the clinical-ultrasonic signature and ultrasomics signature appear to exhibit overfitting tendencies in the training cohort, as indicated in Table 3. Nevertheless, these two models demonstrated a good fit in the test cohort. To address this issue, an ultrasomics nomogram model was employed to combine the ultrasomics signature and clinical-ultrasonic signature, resulting in the most optimal performance achieved by constructing a more precise and consistent prediction model. The ROC curves of these three models, namely the clinical-ultrasonic signature model, ultrasomics signature model, and ultrasomics nomogram, are exhibited in Figure 7 for both the training and test cohorts. To assess the effectiveness of these models, the Delong test was conducted. Fortunately, the AUC of the ultrasomics nomogram demonstrated significant superiority over the ultrasomics signature model and clinical-ultrasonic signature model in the testing cohort (Table 4), indicating that this ultrasomics nomogram (Figure 8) exhibited the highest level of diagnostic efficacy.

Figure 7

Figure 7 The ROC curves of the ultrasomics nomogram model (abbreviated “Nomogram”), clinical-ultrasonic signature (abbreviated “Clinic Signature”), and ultrasomics signature (abbreviated “Ultra Signature”) in the (A) training cohort and (B) test cohort, respectively.

Table 4

Table 4 The results of Delong test and Hosmer-Lemeshow test.

Figure 8

Figure 8 The ultrasomics nomogram model predicts PNET based on clinical-ultrasonic signature (abbreviated “Clinic_Sig”) and ultrasomics signatures (abbreviated “Ultra_Sig”) simultaneously. The nomogram is used by summing all points identified on the scale for each variable. The total points projected on the bottom scales indicate the probabilities of PNET.

The calibration curves of the ultrasomics nomogram performed respectable consistency between the predicted and observed PNET in both the training and test cohorts. The results of the H-L test illustrated that the ultrasomics nomogram had greater prediction accuracy than the other models (Table 4). The calibration curves for the training and test cohorts are displayed in Figure 9.

Figure 9

Figure 9 The calibration curves for the ultrasomics nomogram model (abbreviated “Nomogram”), clinical-ultrasonic signature (abbreviated “Clinic Signature”), and ultrasomics signature (abbreviated “Ultra Signature”) in the (A) training cohort and (B) test cohort, respectively.

Additionally, DCA curves were conducted to assess the performance of each model, and the results are depicted in Figure 10. The ultrasomics nomogram demonstrated a significant net benefit for patients receiving intervention based on its prediction probability compared to the hypothetical scenarios where no prediction model was accessible, such as the treat-all or treat-none schemes. Furthermore, the ultrasomics nomogram exhibited elevated values compared to those of the other signatures in both the training and test cohorts. Consequently, this ultrasomics nomogram has been shown to potentially enhance the clinical efficacy of predicting PNET, before surgery and EUS-FNA procedures.

Figure 10

Figure 10 The DCA curves for the ultrasomics nomogram model (abbreviated “Nomogram”), clinical-ultrasonic signature (abbreviated “Clinic Signature”), and ultrasomics signature (abbreviated “Ultra Signature”) in the (A) training cohort and (B) test cohort, respectively.

Discussion

The ability of EUS to perform comprehensive ultrasound scans of the entire pancreas from the stomach and duodenum, while maintaining proximity and minimal interference, has been widely acknowledged. This capability enables the generation of high-resolution images and facilitates the visualization of intricate anatomical features. Furthermore, the potential for tissue acquisition through EUS-FNA enhances the diagnostic accuracy of EUS, positioning it as one of the most reliable approaches for diagnosing PNET. Moreover, endoscopic ultrasound-fine needle aspiration/biopsy (EUS-FNA/B) is a precise method for determining the pathological grading of PNET, with a pooled estimate rate for overall concordance of 80.3% (95% CI 75.6–85.1) (17). Based on a comprehensive analysis of ten prior studies involving a total of 261 participants, EUS had been shown to demonstrate a commendable average predictive accuracy of 90% (with a range of 77–100%) in the diagnosis of PNET (18). The Chinese Neuroendocrine Tumor (CSNET) collaboration has reached a consensus on the treatment strategy for scattered solitary small NF-PNET, recommending surgical resection and lymph node dissection for most <2cm NF-PNET (19). According to Manta et al, patients diagnosed with pathologically confirmed PNET underwent simultaneous preoperative CT and EUS examinations. The overall detection rate of CT for detecting lesions was 64%. However, for lesions with a diameter less than 10 mm and a range of 11–20 mm, the CT missed diagnosis rate was remarkably high at 68% and 15% respectively. Conversely, EUS had demonstrated a successful detection rate of approximately 47.5% for small pancreatic PNET lesions measuring between 4 and 10 mm in diameter (20). Preoperative EUS imaging for F-PNET can accurately assess the correlation and proximity of the lesion to the main pancreatic duct, thereby playing a pivotal role in determining the appropriate surgical approach, whether it is local or radical resection (21). In some PNET patients, MEN1 may be complicated, and in these patients, multiple small pancreatic lesions are common. Given the limited efficacy of conventional CT and MRI for identifying these small lesions, the utilization of EUS and contrast-enhanced EUS is highly recommended (22). Consequently, EUS plays a crucial role in the detection, diagnosis, and operation strategy selection of PNET.

As previously elucidated, our study provides evidence that patients with PNET may exhibit a younger age profile, and the lesions are more likely to have a shorter diameter and regular shape, than those with pancreatic cancer. Furthermore, the variables of age, maximum diameter, and shape were found to be independent predictors of PNET. In line with our findings, Iordache S observed a statistically significant difference in the age of patients with pancreatic adenocarcinoma compared to those with PNET (62.40 ± 11.24 vs 62.40 ± 11.24, P = 0.0119) (23). PNET was commonly distinguished by low-intensity echoes, well-defined borders, regular roundness in shape, vascularization, and uniform internal echo patterns (24). Furthermore, it has been reported that the median volume of PNET was notably smaller than that of pancreatic adenocarcinomas (25). In summary, the characteristics of the clinical and EUS ultrasonic features observed in this study were extremely consistent with those of previous reports in the literature, further validating the reliability of the research results.

The acquisition of technical expertise via EUS was a demanding endeavor, characterized by a steep and intricate learning curve (26). According to the American Society for Gastrointestinal Endoscopy (ASGE), a comprehensive EUS practice should encompass a minimum of two years of routine gastrointestinal endoscopy experience, coupled with at least one year of specialized training in biliopancreatic EUS (27). Contrary to the normal US and digestive endoscopy, EUS necessitated a higher level of proficiency in endoscopic techniques, exceptional spatial perception, extensive diagnostic experience, and a comprehensive understanding of anatomical knowledge by physicians. These factors contributed to the variability in recognizing and interpreting macroscopic imaging characteristics of EUS images, thereby limiting the specificity and sensitivity of EUS diagnosis (28). Cystic changes were observed in 8–17% of PNET patients and manifest as unilocular, septated, microcystic, or mixed solid and cystic formations. Nevertheless, these cystic alterations lacked specificity, rendering differentiation through radiology or EUS alone a challenging task (29). The subjective interpretation of EUS image features, including tumor size, shape, border, and vessel invasion, relied heavily on the operator’s experience, resulting in inadequate homogeneity. Therefore, there is a pressing need for a novel approach utilizing EUS imaging that offers enhanced objectivity and accuracy in predicting PNET.

Radiomics enables the extraction of multidimensional data from medical images that surpass human visual assessment. By converting medical images into extractable data, this approach facilitates the design of classification models utilizing machine learning algorithms. Consequently, this approach enhanced the identification of various tumor types with increased reliability and objectivity (6, 30). Recently, the academic community has shown increasing interest in the exploration of EUS-based ultrasomics. In a multicenter study, Li XY demonstrated that the integration of machine learning algorithms with EUS ultrasomics features enabled the development of an effective classification model for assessing the malignancy grade of gastrointestinal stromal tumors (GISTs) (6). Gu et al. successfully devised a deep-learning ultrasomics model utilizing EUS images to diagnose pancreatic ductal adenocarcinoma. This model effectively mitigated the diagnostic disparity among ultrasound endoscopy physicians of varying expertise levels, thereby enhancing the precision of their diagnoses (31). The EUS-based ultrasomics model was developed to accurately distinguish between gastric GISTs, smooth muscle tumors, and nerve sheath tumors (32). Regrettably, the literature lacked published studies that have utilized EUS imaging ultrasomics to diagnose and predict PNET.

Our study revealed that a comprehensive set of 107 EUS imaging ultrasomics features were initially extracted. Subsequently, through rigorous statistical analyses including variance analysis, correlation analysis, and LASSO regression analysis, a subset of 6 ultrasomics features were identified as highly significant and definitively associated with PNET. Multiple mainstream machine learning algorithms were used simultaneously to construct the most appropriate two-class prediction model for distinguishing PNET from pancreatic cancer, to overcome the limitations of single algorithms. In this instance, the RF algorithm seemed to demonstrate superior accuracy and consistency and was subsequently applied for further model development. In the training cohort, both the ultrasomics signature model and clinical-ultrasonic signature model based on the RF algorithm achieved respectable AUCs. However, the effectiveness of the ultrasomics signature model was inferior to that of the clinical-ultrasonic signature model. Interestingly, we constructed an ultrasomics nomogram by combining the ultrasomics signature and clinical-ultrasonic signature as described previously, with a significant improvement in prediction efficiency in the testing cohort. Moreover, the DeLong test and H-L test have confirmed the validity and accuracy of this ultrasomics nomogram.

The RF algorithm, an ensemble learning technique that integrates a variety of decision trees, stands out for its exceptional accuracy compared to other existing algorithms and has the potential to enhance selection during classification prediction (33). Our results illustrated that although the clinical-ultrasonic signature model owned relatively high effectiveness, the ultrasomics nomogram achieved better predictive performance. To the best of our knowledge, this study represents the initial demonstration that the EUS-based ultrasomics nomogram significantly and efficiently improved the prediction of PNET. Given the higher detection rate of PNET and the superior preoperative lesion detail assessment capabilities of EUS compared to CT and MRI, these findings are expected to contribute to the advancement of EUS utilization in the management of PNET.

Despite the notable efficacy of the ultrasomics nomogram utilizing EUS imaging, this study is subject to certain limitations. Notably, this retrospective analysis was conducted within a single institution based on different EUS devices, potentially introducing selection and systematic bias. Additionally, all boundary definitions were derived from manual segmentation in image segmentation, and bias was inevitable (34). Furthermore, it is essential to acknowledge that this study utilized only conventional EUS images, disregarding the potential added value of contrast-enhanced EUS and elastography EUS techniques (22, 35, 36). Hence, it is imperative to undertake future studies on EUS-based ultrasomics for PNET that encompass multiple centers, large sample sizes, prospective designs, and multimodal approaches based on the same equipment. Another limitation of this study was that all the ultrasomics features were assessed in two instead of three dimensions. We expect that extending this work to three dimensions based on EUS videos may yield improved model accuracy and representability in the future. Moreover, regarding the critical value of pathological grade and tumor size in the diagnosis, treatment, and prognosis of PNET (37), the predictive value of EUS-based ultrasomics for PNET pathological grading and small tumors (<15mm) also needs to be further evaluated. A concurrent EUS-FNA subgroup analysis to further clarify the accuracy of EUS-based ultrasomics would also be of high value. Additionally, in clinical practice, the artifacts and noise of EUS images are not always avoided, so integrating deep learning techniques based on automatic image segmentation could mitigate bias and enhance the generalizability and applicability of the models.

Conclusion

In summary, an efficient EUS-based ultrasomics nomogram, incorporating clinical-ultrasonic and ultrasomics signatures, was proposed and validated for predicting pancreatic cancer and PNET. These outcomes provide potential novel insight into improving clinical prediction and treatment strategies for PNET.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary Material. Further inquiries can be directed to the corresponding authors.

Ethics statement

The studies involving humans were approved by the Medical Ethics Committee of The First Affiliated Hospital of Guangxi Medical University (No. 2023-E732-01). The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study. Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.

Author contributions

SM: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing. CH: Formal analysis, Software, Visualization, Writing – original draft, Writing – review & editing. YW: Writing – original draft, Writing – review & editing. HZ: Formal analysis, Investigation, Methodology, Resources, Writing – original draft. HW: Writing – original draft, Writing – review & editing. HQ: Writing – original draft, Writing – review & editing. HJ: Funding acquisition, Methodology, Resources, Writing – original draft, Writing – review & editing. SQ: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This study was supported by the Scientific Research Project of Liuzhou People’s Hospital affiliated with Guangxi Medical University (grant numbers: lry202309 and lry202311), the Scientific Research Project of Guangxi Health Commission (grant number: Z-B20231296, Z-B20231339).

Acknowledgments

We appreciate the Onekey AI platform and its developers.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fonc.2024.1359364/full#supplementary-material

Supplementary Figure 1 | The correlation coefficient matrix of ultrasomics features.

Supplementary Figure 2 | The violin plot of each ultrasomics feature along with their corresponding p-value results.

Supplementary Figure 3 | The DCA curves of RF ultrasomics models in the training (A) and test (B) cohorts.

Glossary

References

1. Scott AT, Weitz M, Breheny PJ, Ear PH, Darbro B, Brown BJ, et al. Gene expression signatures identify novel therapeutics for metastatic pancreatic neuroendocrine tumors. Clin Cancer Res. (2020) 26:2011–21. doi: 10.1158/1078-0432.CCR-19-2884

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Song Y, Gao S, Tan W, Qiu Z, Zhou H, Zhao Y. Multiple machine learnings revealed similar predictive accuracy for prognosis of PNETs from the surveillance, epidemiology, and end result database. J Cancer. (2018) 9:3971–8. doi: 10.7150/jca.26649

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Keutgen XM, Kumar S, Gara SK, Boufraqech M, Agarwal S, Hruban RH, et al. Transcriptional alterations in hereditary and sporadic nonfunctioning pancreatic neuroendocrine tumors according to genotype. Cancer. (2018) 124:636–47. doi: 10.1002/cncr.31057

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Li D, Zhou Y, Liu Y, Lin Y, Yu M, Lu X, et al. Decreased expression of LKB1 predicts poor prognosis in pancreatic neuroendocrine tumor patients undergoing curative resection. OncoTargets Ther. (2018) 11:1259–65. doi: 10.2147/OTT

CrossRef Full Text | Google Scholar

5. Frost M, Lines KE, Thakker RV. Current and emerging therapies for PNETs in patients with or without MEN1. Nat Rev Endocrinol. (2018) 14:216–27. doi: 10.1038/nrendo.2018.3

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Li X, Jiang F, Guo Y, Jin Z, Wang Y. Computer-aided diagnosis of gastrointestinal stromal tumors: a radiomics method on endoscopic ultrasound image. Int J Comput assisted Radiol Surg. (2019) 14:1635–45. doi: 10.1007/s11548-019-01993-3

CrossRef Full Text | Google Scholar

7. Cigrovski Berković M, Catela Ivković T, Marout J, Zjačić-Rotkvić V, Kapitanović S. Interleukin 1β gene single-nucleotide polymorphisms and susceptibility to pancreatic neuroendocrine tumors. DNA Cell Biol. (2012) 31(4):531–6. doi: 10.1089/dna.2011.1317

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Melita G, Pallio S, Tortora A, Crinò SF, Macrì A, Dionigi G. Diagnostic and interventional role of endoscopic ultrasonography for the management of pancreatic neuroendocrine neoplasms. J Clin Med. (2021) 10(12):2638. doi: 10.3390/jcm10122638

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Hofland J, Falconi M, Christ E, Castaño JP, Faggiano A, Lamarca A, et al. European Neuroendocrine Tumor Society 2023 guidance paper for functioning pancreatic neuroendocrine tumour syndromes. J Neuroendocrinol. (2023) 35:e13318. doi: 10.1111/jne.13318

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Parasher G, Wong M, Rawat M. Evolving role of artificial intelligence in gastrointestinal endoscopy. World J Gastroenterol. (2020) 26:7287–98. doi: 10.3748/wjg.v26.i46.7287

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Zhu HB, Zhu HT, Jiang L, Nie P, Hu J, Tang W, et al. Radiomics analysis from magnetic resonance imaging in predicting the grade of nonfunctioning pancreatic neuroendocrine tumors: a multicenter study. Eur Radiol. (2024) 34(1):90–102. doi: 10.1007/s00330-023-09957-7

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Gu D, Hu Y, Ding H, Wei J, Chen K, Liu H, et al. CT radiomics may predict the grade of pancreatic neuroendocrine tumors: a multicenter study. Eur Radiol. (2019) 29:6880–90. doi: 10.1007/s00330-019-06176-x

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Zhang L, Qi Q, Li Q, Ren S, Liu S, Mao B, et al. Ultrasomics prediction for cytokeratin 19 expression in hepatocellular carcinoma: A multicenter study. Front Oncol. (2022) 12:994456. doi: 10.3389/fonc.2022.994456

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Tu C, Shen H, Liu R, Wang X, Li X, Yuan X, et al. Myocardial extracellular volume derived from contrast-enhanced chest computed tomography for longitudinal evaluation of cardiotoxicity in patients with breast cancer treated with anthracyclines. Insights into Imaging. (2022) 13:85. doi: 10.1186/s13244-022-01224-5

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Yin R, Jiang M, Lv WZ, Jiang F, Li J, Hu B, et al. Study processes and applications of ultrasomics in precision medicine. Front Oncol. (2020) 10:1736. doi: 10.3389/fonc.2020.01736

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Fornacon-Wood I, Mistry H, Ackermann CJ, Blackhall F, McPartlin A, Faivre-Finn C, et al. Reliability and prognostic value of radiomic features are highly dependent on choice of feature extraction platform. Eur Radiol. (2020) 30:6241–50. doi: 10.1007/s00330-020-06957-9

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Tacelli M, Bina N, Crinò SF, Facciorusso A, Celsa C, Vanni AS, et al. Reliability of grading preoperative pancreatic neuroendocrine tumors on EUS specimens: a systematic review with meta-analysis of aggregate and individual data. Gastrointest Endosc. (2022) 96(6):898–908.e823. doi: 10.1016/j.gie.2022.07.014

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Sundin A, Vullierme MP, Kaltsas G, Plöckinger U. ENETS Consensus Guidelines for the Standards of Care in Neuroendocrine Tumors: radiological examinations. Neuroendocrinology. (2009) 90:167–83. doi: 10.1159/000184855

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Yang G, Ji M, Chen J, Chen R, Chen Y, Fu D, et al. Surgery management for sporadic small (ma cm), non-functioning pancreatic neuroendocrine tumors: a consensus statement by the Chinese Study Group for Neuroendocrine Tumors (CSNET). Int J Oncol. (2017) 50:567–74. doi: 10.3892/ijo.2016.3826

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Zilli A, Arcidiacono PG, Conte D, Massironi S. Clinical impact of endoscopic ultrasonography on the management of neuroendocrine tumors: lights and shadows. Digestive liver Dis. (2018) 50:6–14. doi: 10.1016/j.dld.2017.10.007

CrossRef Full Text | Google Scholar

21. Giuliani T, Marchegiani G, Girgis MD, Crinò SF, Muthusamy VR, Bernardoni L, et al. Endoscopic placement of pancreatic stent for "Deep" pancreatic enucleations operative technique and preliminary experience at two high-volume centers. Surg endoscopy. (2020) 34:2796–802. doi: 10.1007/s00464-020-07501-y

CrossRef Full Text | Google Scholar

22. Costache MI, Cazacu IM, Dietrich CF, Petrone MC, Arcidiacono PG, Giovannini M, et al. Clinical impact of strain histogram EUS elastography and contrast-enhanced EUS for the differential diagnosis of focal pancreatic masses: A prospective multicentric study. Endoscopic ultrasound. (2020) 9:116–21. doi: 10.4103/eus.eus_69_19

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Iordache S, Angelescu R, Filip MM, Costache MI, Popescu CF, Gheonea DI, et al. Power Doppler endoscopic ultrasound for the assessment of pancreatic neuroendocrine tumors. Endoscopic ultrasound. (2012) 1:150–5. doi: 10.7178/eus

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Di Leo M, Poliani L, Rahal D, Auriemma F, Anderloni A, Ridolfi C, et al. Pancreatic neuroendocrine tumours: the role of endoscopic ultrasound biopsy in diagnosis and grading based on the WHO 2017 classification. Digestive Dis (Basel Switzerland). (2019) 37:325–33. doi: 10.1159/000499172

CrossRef Full Text | Google Scholar

25. Baxi AC, Jiang Q, Hao J, Yang Z, Woods K, Keilin S, et al. The effect of solid pancreatic mass lesions on pancreatic duct diameter at endoscopic ultrasound. Endoscopic ultrasound. (2017) 6:103–8. doi: 10.4103/2303-9027.204812

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Wani S, Keswani R, Hall M, Han S, Ali MA, Brauer B, et al. A prospective multicenter study evaluating learning curves and competence in endoscopic ultrasound and endoscopic retrograde cholangiopancreatography among advanced endoscopy trainees: the rapid assessment of trainee endoscopy skills study. Clin Gastroenterol Hepatol. (2017) 15(11):1758–67.e1711. doi: 10.1016/j.cgh.2017.06.012

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Cho CM. Training in endoscopy: endoscopic ultrasound. Clin endoscopy. (2017) 50:340–4. doi: 10.5946/ce.2017.067

CrossRef Full Text | Google Scholar

28. Zhang J, Zhu L, Yao L, Ding X, Chen D, Wu H, et al. Deep learning-based pancreas segmentation and station recognition system in EUS: development and validation of a useful training tool (with video). Gastrointest Endosc. (2020) 92(4):874–85.e873. doi: 10.1016/j.gie.2020.04.071

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Lee LS. Diagnosis of pancreatic neuroendocrine tumors and the role of endoscopic ultrasound. Gastroenterol Hepatol (N Y). (2010) 6(8):520–2.

PubMed Abstract | Google Scholar

30. Gillies RJ, Kinahan PE, Hricak H. Radiomics: images are more than pictures, they are data. Radiology. (2016) 278:563–77. doi: 10.1148/radiol.2015151169

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Gu J, Pan J, Hu J, Dai L, Zhang K, Wang B, et al. Prospective assessment of pancreatic ductal adenocarcinoma diagnosis from endoscopic ultrasonography images with the assistance of deep learning. Cancer. (2023) 129:2214–23. doi: 10.1002/cncr.34772

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Zhang XD, Zhang L, Gong TT, Wang ZR, Guo KL, Li J, et al. A combined radiomic model distinguishing GISTs from leiomyomas and schwannomas in the stomach based on endoscopic ultrasonography images. J Appl Clin Med Phys. (2023) 24:e14023. doi: 10.1002/acm2.14023

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Zheng Y, Zhou D, Liu H, Wen M. CT-based radiomics analysis of different machine learning models for differentiating benign and Malignant parotid tumors. Eur Radiol. (2022) 32:6953–64. doi: 10.1007/s00330-022-08830-3

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Lohmann P, Bousabarah K, Hoevels M, Treuer H. Radiomics in radiation oncology-basics, methods, and limitations. Strahlentherapie und Onkologie: Organ der Deutschen Rontgengesellschaft …. (2020) 196:848–55. doi: 10.1007/s00066-020-01663-3

CrossRef Full Text | Google Scholar

35. Franchellucci G, Andreozzi M, Carrara S, De Luca L, Auriemma F, Paduano D, et al. Contrast enhanced EUS for predicting solid pancreatic neuroendocrine tumor grade and aggressiveness. Diagnostics (Basel). (2023) 13(2):239. doi: 10.3390/diagnostics13020239

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Tang A, Tian L, Gao K, Liu R, Hu S, Liu J, et al. Contrast-enhanced harmonic endoscopic ultrasound (CH-EUS) MASTER: A novel deep learning-based system in pancreatic mass diagnosis. Cancer Med. (2023) 12:7962–73. doi: 10.1002/cam4.5578

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Crinò SF, Conti Bellocchi MC, Bernardoni L, Manfrin E, Parisi A, Amodio A, et al. Diagnostic yield of EUS-FNA of small (≤15 mm) solid pancreatic lesions using a 25-gauge needle. Hepatobiliary Pancreat Dis Int. (2018) 17(1):70–4. doi: 10.1016/j.hbpd.2018.01.010

PubMed Abstract | CrossRef Full Text | Google Scholar