Development of a prognostic nomogram for ocular melanoma: a SEER population-based study (2000–2021)

Introduction: Ocular melanoma (OM) is a rare but lethal subtype of melanoma. This study develops a prognostic nomogram for OM using machine learning and internal validation techniques, aiming to improve prognosis prediction and clinical decision-making.

Methods: Independent prognostic variables were identified using univariate and multivariate COX proportional hazard regression models. Significant variables were then incorporated into the nomogram. The predictive accuracy of the nomogram was evaluated using receiver operating characteristic (ROC) curves, calibration plots, decision curve analysis (DCA), and 10-fold cross-validation. The performance of the nomogram was compared with that of a machine learning model.

Results: Thirteen variables, including age, sex, tumor site, histologic subtype, stage, basal diameter size, tumor thickness, liver metastasis, first malignant primary indicator, marital status, and treatment modalities (surgery/radiotherapy/chemotherapy) were identified as independent prognostic factors for overall survival (OS) and were included in the nomogram (all P < 0.05). The nomogram showed a concordance index of 0.712. The areas under the curve (AUC) for predicting 3-, 5-, and 10-year survival rates were 0.749, 0.734, and 0.730, respectively. Calibration plots for 3-, 5-, and 10-year survival were in close agreement with the ideal predictions, and DCA indicated a superior net benefit. The average AUC from 10-fold cross-validation was 0.725. The machine-learning model identified liver metastasis as the most significant predictor of survival, followed by age, radiotherapy, stage, and other factors that were incorporated into the nomogram. The machine-learning model achieved a predictive AUC score of 0.750.

Conclusions: A robust nomogram incorporating 13 significant clinicopathological variables was developed. The combined use of ROC curve analysis, calibration plots, DCA, 10-fold cross-validation, and machine learning confirmed the strong predictive performance of the nomogram for survival outcomes in patients with OM.

1 Introduction

Ocular melanoma (OM) is a rare subtype of melanoma; the incidence rate is about 6.2 per million, accounting for ~5% of all melanoma cases (1). Uveal melanoma (UM), which includes melanomas of the choroid, ciliary body, and iris, is the predominant anatomical site of OM, accounting for ~85% of these cases (2). Despite its rare occurrence, OM is highly lethal owing to its aggressive nature and high potential for metastasis (3). Although primary melanoma can be treated effectively, ~30% of patients experience recurrence that involves metastatic disease. The prognosis of OM is related to many factors, such as clinical, pathological, and genomic characteristics (4–6). However, a reliable prognostic model for predicting the outcome of OM is currently lacking.

Given the high prevalence of UM among OM, delineating independent prognostic indicators and formulating prognostic frameworks for UM is necessary (7–9). Nonetheless, the absence of an optimal prognostic prediction model encompassing all OM subtypes underscores the pressing need for a robust predictive model to elucidate OM prognosis and inform clinical decision-making. Nomograms, which are graphical instruments that assign distinct points to each variable within a predictive model, facilitate the translation of cumulative points into corresponding survival probabilities.

Machine-learning algorithms have emerged as innovative tools that are extensively employed in cancer research. Among them, eXtreme Gradient Boosting (XGBoost) as a prediction gradient boosting tree stands out as a notable ensemble learning technique (10). This method incrementally enhances prediction efficacy by iteratively training a sequence of decision trees. Owing to their ability to manage structured data and capture non-linear associations, gradient-boosting trees outperform random forests in terms of prediction accuracy. SHapley Additive exPlanations (SHAP) constitute a method devised to elucidate the prediction outcomes derived from machine learning models (11). By integrating these two methodologies, we can assess the predictive effectiveness of the model in conjunction with a profound understanding of how each feature shapes the prediction outcome.

In this study, we developed a prognostic prediction nomogram tailored specifically for patients with OM. Given the rarity of OM, external validation of the model using a substantial dataset remains challenging. To comprehensively assess the predictive probability of our model, we used traditional evaluation methods, including receiver operating characteristic (ROC) curves, calibration curves, and decision curve analysis (DCA), along with internal validation techniques. Additionally, we leveraged advanced methodologies, namely, XGBoost, in conjunction with SHAP to conduct a multifaceted evaluation, compare models, and discern the efficacy of the established prognostic model across various dimensions.

2 Materials and methods

2.1 Study patients

Cohort information for all patients was extracted from the National Cancer Institute Surveillance, Epidemiology, and End Results (SEER) database between 2000 and 2021. SEER^*Stat software version 8.4.3 was used for data selection which covers ~26.5% of the U.S. population from 17 registries. OMs were defined by the International Classification of Diseases for Oncology, Third Edition (ICD-O-3) site code, C69.0-69.6 or C69.8-69.9, and ICD-O-3 histology codes 8720-8790.

The inclusion criteria were as follows: (1) patients diagnosed with primary OM between 2000 and 2021; (2) ICD-O-3 site code of C69.0 (conjunctiva), C69.1 (cornea, not otherwise specified [NOS]), C69.2 (retina), C69.3 (choroid), C69.4 (ciliary body and iris), C69. 5(lacrimal gland), C69.6 (orbit, NOS), C69.8 (overlapping lesion of eye and adnexa), and C69.9 (eye, NOS); (3) ICD-O-3 histology code 8770 (mixed epithelioid and spindle cell melanomas), 8771 (epithelioid cell melanomas), 8772-8774 (spindle-cell melanomas), 8720 (malignant melanoma, NOS) and other (including 8721-8723 [nodular melanoma, balloon cell melanoma and malignant melanoma with regression], 8730 [amelanotic melanoma], 8740-8746 [malignant melanoma in junctional nevus, malignant melanoma in precancerous melanosis, lentigo malignant melanoma, superficial spreading melanoma, acral lentiginous melanoma, desmoplastic melanoma, and mucosal lentiginous melanoma], and 8761 [malignant melanoma in giant pigmented nevus]); and (4) patients with localized, regional and distant tumors defined using SEER staging system (SEER historic stage A and combined summary stage). Patients with unknown survival data or survival duration < 1 month, unknown laterality, SEER stage, race and marital status were excluded. A flowchart of patient selection is shown in Figure 1. Because the study used a de-identified and publicly available database, ethical approval was not required. The design and analysis procedures were performed in accordance with the principles outlined in the Declaration of Helsinki. Additional information regarding the SEER program can be found on the SEER website (https://seer.cancer.gov).

Figure 1

Figure 1. Flow chart of cases selection from SEER database.

2.2 Study variables

The variables used in the SEER database included age, sex, race, site, laterality, histological subtype, combined summary stage, SEER historic stage A, surgery, radiation, chemotherapy, measured basal diameter, measured thickness, metastasis to the liver, metastasis to the lung, first malignant primary indicator (FMPI), marital status, survival months, and vital status. The overall survival (OS) was designated as the outcome-predicting variable, which was measured as the time from diagnosis to death due to any cause.

2.3 Establish and validate the nomogram model

To build a nomogram model for predicting prognostic risk factors for ocular melanoma, we initially used a COX proportional hazards regression model. The independent variables were analyzed using univariate COX proportional hazards regression, and those with P < 0.05 were included in multivariate COX proportional hazards regression. Subsequently, statistically significant indicators were included in the nomogram model generation. Additionally, hazard ratios (HR) and matched 95% confidence intervals (CI) were recorded.

By incorporating the selected indicator factors, we developed a nomogram to predict the 3-, 5-, and 10-year survival for patients with OM. The concordance index (C-index) served as a metric to gauge the discriminatory capacity of the model, ranging from 0.5 to 1.0. A value of 0.5 signifies random performance, whereas 1.0 denotes perfect alignment between the model predictions and actual outcomes. Additionally, ROC curves and their associated area under the curve (AUC) were used to elucidate the balance between true-positive and false-positive rates across various thresholds. Moreover, calibration curves were used to assess the agreement between the predicted and observed probabilities at the 3-, 5-, and 10-year intervals. DCA was conducted to evaluate the clinical relevance of the model by examining its net benefit across different decision thresholds.

The 10-fold cross-validation involved dividing the dataset into 10 equally-sized subsets. In each iteration, one subset was used for testing, and the remaining nine subsets were used for training. This process was repeated 10 times to ensure that each subset was tested at least once.

2.4 Statistical analysis

The X-tile software (Yale University, USA, Version 3.6.1) was used to determine the optimal cutoff value for continuous variables by artificially dividing them into distinct groups. The highest chi-square values calculated by Kaplan–Meier survival analysis and the logarithmic rank test were used as the optimal cutoff values. Based on the results, we divided the age into ≤ 61 years and >61 years. The tumor basal diameter was divided into ≤ 14.9 mm and >14.9 mm, and the measured thickness was divided into ≤ 5.1 mm and >5.1 mm. Patient characteristics, including numbers and proportions, were analyzed using descriptive statistical methods. The Kaplan–Meier method was used to analyze the OS and corresponding survival curves.

XGBoost is a powerful gradient boosting tree algorithm that can be combined with SHAP to obtain more accurate and interpretable prediction models. To analyze the data produced by the machine learning models, we used an explanation model called SHAP to address the problem of black-box predictions. By calculating the SHAP value, the SHAP explanation model measured the impact of each input feature on the predicted output. The values were then used to prioritize the features and visualize important associations. The importance and predictive power of the prognostic predictor increased with higher values. The SHAP analysis utilized the SHAPforxgboost library (V.0.1.3). Finally, the results of the machine learning algorithm were compared with the indicators in the nomogram.

All statistical analyses were performed using RStudio software (V.4.3.1; R Core Team, Vienna, Austria). All statistical tests were two-tailed, and P < 0.05 indicated statistical significance.

3 Results

3.1 Patient characteristics

This study included 9,120 patients [4,329 women (47.5%) and 4,791 men (52.5%)] with primary OM. The demographic and clinicopathological features and treatment information of patients with OM are summarized in Table 1. Among them, the age at OM diagnosis was >61 years in 4,980 patients (54.6%), and the majority of patients were Caucasians (97.4%). The choroid was the most common primary OM site (78.3%), followed by the ciliary body (10.8%). All patients had unilateral lesions with similar rates on both the left and right sides. Based on the histologic subtypes, 7,150 (78.4%) patients were included in the NOS category, 146 (1.6%) were classified as having other types combined as NOS/Other group (80%), 836 (9.2%) had spindle cell melanoma, 715 (7.8%) had mixed epithelioid and spindle cell melanoma, and 273 (3%) had epithelioid cell melanoma. We combined the Combined Summary Stage (2004+) and SEER historical stage A (1973–2015) into the SEER stage system. Localized, regional, and distant tumors were present in 91%, 6.4%, and 2.6% of patients, respectively. Basal diameter sizes ≤ and >14.9 mm were present in 2,950 (32.3%) and 796 (8.7%) patients, respectively. Tumor thickness >5.1 mm and ≤ 5.1 mm were present in 16.4% and 28.3% of patients, respectively. The rate of metastasis in the liver (1.2 %) was higher than that in the lungs (0.5%). Most patients had OM as their FMPI, and most were married (65.7%). In terms of treatment, 3,271 (35.9%), 6,181 (67.8%), and 247 (2.7%) patients had undergone surgery, radiotherapy, and chemotherapy, respectively. The median survival time was 132 months (Figure 2).

Table 1

Table 1. Baseline demographics and clinical characteristics of patients (n = 9,120).

Figure 2

Figure 2. The overall survival curve in all ocular melanoma patients. The median survival time was 132 months.

3.2 Prognostic factor selection

The analysis included 16 independent variables: age, sex, race, site, laterality, histological subtype, stage, basal diameter, thickness, liver metastasis, lung metastasis, FMPI, marital status, surgery, radiotherapy, and chemotherapy. To select the independent prognostic factors for OM patients, we used the univariate and multivariate COX proportional hazards regression models (Table 2). In the univariate analysis, age >61 years (vs. age ≤ 61 years: HR = 2.680, 95% CI = 2.498–2.876, P < 0.001); sex male (vs. female: HR = 1.156, 95% CI = 1.084–1.233, P < 0.001); race, Blacks (vs. Caucasians: HR = 0.836, 95% CI = 0.668–1.387, P = 0.836), other (vs. Caucausians: HR = 1.179, 95% CI = 0.929–1.496, P = 0.176); site, ciliary body, and iris (vs. choroid: HR = 1.095, 95% CI = 0.989–1.212, P = 0.080), conjunctiva (vs. choroid: HR = 0.910, 95% CI = 0.791–1.046, P = 0.185), other (vs. choroid: HR = 1.872, 95% CI = 1.653–2.120, P < 0.001); laterality, right (vs. left: HR = 0.989, 95% CI = 0.927–1.055, P = 0.739); histological subtype, epithelioid (vs. spindle: HR = 2.801, 95% CI = 2.325–3.373, P < 0.001), mixed (vs. spindle: HR = 2.228, 95% CI = 1.924–2.579, P < 0.001), NOS/Other (vs. spindle: HR = 1.203, 95% CI = 1.07–1.352, P = 0.002); stage, regional (vs. localized: HR = 1.814, 95% CI = 1.615–2.039, P < 0.001), distant (vs. localized: HR = 7.321, 95% CI = 6.335–8.461, P < 0.001); basal diameter size, >14.9 mm (vs. ≤ 14.9 mm: HR = 2.618, 95% CI = 2.294–2.988, P < 0.001), unknown (vs. ≤ 14.9 mm: HR = 1.544, 95% CI = 1.412–1.687, P < 0.001); tumor thickness, > 5.1 mm (vs. ≤ 5.1 mm: HR = 2.016, 95% CI = 1.784–2.278, P < 0.001), unknown (vs. ≤ 5.1 mm: HR = 1.664, 95% CI = 1.511–1.833, P < 0.001); liver metastasis (vs. no liver metastasis: HR = 12.293, 95% CI = 9.905–15.255, P < 0.001), unknown (vs. no liver metastasis: HR = 1.161, 95% CI = 1.081–1.247, P < 0.001); lung metastasis (vs. no lung metastasis: HR = 8.795, 95% CI = 6.257–12.365, P < 0.001), unknown (vs. no lung metastasis: HR = 1.125, 95% CI = 1.048–1.208, P = 0.001); FMPI (vs. non-FMPI: HR = 0.574, 95% CI = 0.531–0.621, P < 0.001); marital status, unmarried (vs. married: HR = 0.996, 95% CI = 0.903–1.097, P = 0.929), divorced (vs. married: HR = 1.223, 95% CI = 1.090–1.372, P = 0.001), widowed (vs. married: HR = 2.121, 95% CI = 1.940–2.320, P < 0.001); surgery (vs. no/unknown surgery: HR = 1.641, 95% CI = 1.539–1.751, P < 0.001); radiotherapy (vs. no/unknown radiotherapy: HR = 0.592, 95% CI = 0.555–0.632, P < 0.001); chemotherapy (vs. no/unknown chemotherapy: HR = 1.818, 95% CI = 1.532–2.158, P < 0.001). We found that OM patients aged >61 years, predominantly men with lesions at other sites other than choroid, non-spindle types, non-localized lesions, basal diameter size >14.9 mm and unknown, tumor thickness >5.1 mm and unknown, metastasis to the liver and unknown, metastasis to the lung and unknown, marital status (divorced and widowed), and those who had undergone surgery/chemotherapy had a higher mortality risk. However, with OM as the FMPI, patients who were unmarried and had undergone radiotherapy showed improved OS. Our results indicated that race and laterality had no impact on OS in patients with OM.

Table 2

Table 2. Univariate and multivariate analysis of overall survival for ocular melanoma patients.

We selected variables with P < 0.05 using the multivariate COX proportional hazards regression model. The final screening results of the multivariate analysis are as follows: age, age >61 years (vs. age ≤ 61 years: HR = 2.463, 95% CI = 2.286–2.654, P < 0.001); sex, male (vs. female: HR = 1.249, 95% CI = 1.167–1.337, P < 0.001); site, ciliary body and iris (vs. choroid: HR = 0.992, 95% CI = 0.894–1.101, P = 0.879), conjunctiva (vs. choroid: HR = 0.523, 95% CI = 0.444–0.614, P < 0.001), other (vs. choroid: HR = 1.290, 95% CI = 1.128–1.475, P < 0.001); histological subtype, epithelioid (vs. spindle: HR = 2.337, 95% CI = 1.937–2.819, P < 0.001), mixed (vs. spindle: HR = 1.685, 95% CI = 1.453–1.955, P < 0.001), NOS/other (vs. spindle: HR = 1.520, 95% CI = 1.342–1.721, P < 0.001); stage, regional (vs. localized: HR = 1.436, 95% CI = 1.273–1.620, P < 0.001), distant (vs. localized: HR = 4.108, 95% CI = 3.327–5.072, P < 0.001); basal diameter size, >14.9 mm (vs. ≤ 14.9 mm: HR = 1.777, 95% CI = 1.536–2.055, P < 0.001), unknown (vs. ≤ 14.9 mm: HR = 1.180, 95% CI = 1.012–1.377, P = 0.035); tumor thickness, > 5.1 mm (vs. ≤ 5.1 mm: HR = 1.512, 95% CI = 1.320–1.732, P < 0.001), unknown (vs. ≤ 5.1 mm: HR = 1.249, 95% CI = 1.066–1.463, P = 0.006); liver metastasis (vs. no liver metastasis: HR = 2.058, 95% CI = 1.541–2.749, P < 0.001), unknown (vs. no liver metastasis: HR = 0.975, 95% CI = 0.138–6.905, P = 0.980); lung metastasis (vs. no lung metastasis: HR = 1.217, 95% CI = 0.833–1.777, P = 0.309), unknown (vs. no lung metastasis: HR = 1.083, 95% CI = 0.154–7.630, P = 0.936); FMPI (vs. non-FMPI: HR = 0.708, 95% CI = 0.653–0.767, P < 0.001); marital status, unmarried (vs. married: HR = 1.158, 95% CI = 1.049–1.279, P = 0.004), divorced (vs. married: HR = 1.221, 95% CI = 1.087–1.372, P = 0.001), widowed (vs. married: HR = 1.539, 95% CI = 1.397–1.695, P < 0.001); surgery (vs. no/unknown surgery: HR = 1.227, 95% CI = 1.113–1.354, P < 0.001); radiotherapy (vs. no/unknown radiotherapy: HR = 0.715, 95% CI = 0.648–0.790, P < 0.001); chemotherapy (vs. no/unknown chemotherapy: HR = 1.588, 95% CI = 1.323–1.905, P < 0.001). Briefly, age, sex, site, histological subtype, stage, basal diameter size, tumor thickness, metastasis to the liver, FMPI, marital status, and surgical intervention/radiotherapy/chemotherapy were independent prognostic factors for OS (all P < 0.05).

3.3 Nomogram model for OS prognosis

A nomogram was constructed by incorporating all independent factors to forecast the survival probability of patients with OM at 3, 5, and 10 years (Figure 3). As shown in the figure, the stage of OM had the most significant influence on survival, followed by the tumor site, age, histological subtype, liver metastasis, tumor diameter, chemotherapy, marital status, tumor thickness, FMPI, radiotherapy, sex, and surgery. Each factor in the nomogram was assigned a specific score using a point system. The scores for all factors were summed and a vertical line was drawn to determine the total score, indicating the survival probability of patients with OM at 3, 5, and 10 years.

Figure 3

Figure 3. Prognostic nomogram for predicting the probability of 3-, 5- and 10-year OS in patients with ocular melanoma.

3.4 Model performance

This nomogram had an optimal C-index of 0.712 (95% CI = 0.702–0.722), indicating satisfactory discrimination ability of the survival analysis model. We evaluated the performance of the nomogram in predicting 3-, 5-, and 10-year survival probabilities in patients with OM. ROC curves were used to verify the predictability of the nomogram at different thresholds, and the AUCs were computed for the 3-, 5-, and 10-year survival rates (0.749, 0.734, and 0.730, respectively), and the 95% CI were (0.736–0.763), (0.723–0.747), and (0.719–0.741), respectively (Figure 4). The calibration plots predicting 3-, 5-, and 10-year OS in patients with OM were close to the ideal curves, showing good agreement between the predicted probabilities and the actual observed results (Figure 5). DCA is a novel evaluation tool used to evaluate the net benefit of a predictive model under different thresholds and determine the usefulness of the model in clinical decision-making. The curve demonstrated a superior net benefit, highlighting its significant clinical utility in guiding prognostic assessments (Figure 6). We verified the predictive ability of the proposed model from multiple dimensions.

Figure 4

Figure 4. ROC curves were used to verify the predictability of the nomogram at different thresholds, and the AUCs.

Figure 5

Figure 5. The calibration plots predicting 3- (A), 5- (B), and 10-year (C) survival in patients with OM were close to the ideal curves.

Figure 6

Figure 6. Decision curve of the model in predicting 3- (A), 5- (B), and 10-year (C) prognosis. The blue curve represents the net benefits of the model for predicting outcomes.

3.5 Validation of the nomogram

Using 10-fold cross-validation, we assessed the predictive performance of the nomogram. The ROC curve indicated that the model had good discriminatory power for predicting survival status (Figure 7). The average AUC value obtained from cross-validation was 0.725 (95% CI = 0.711–0.732).

Figure 7

Figure 7. 10-fold cross-validation ROC curve and AUC value.

3.6 Machine learning

The SHAP summary plot is a visualized tool to rank the characteristics in the model and the influence of each feature on the predictive ability of the model. The plot ranked the factors by their importance, with liver metastasis being the most influential, followed by age, radiotherapy, stage, and other factors (Figure 8). These important factors were then incorporated into the nomogram. The multivariate dependence plot demonstrated the interactions between the different variables (Figure 9). Older patients with liver metastases and more advanced tumor stages who had undergone radiotherapy had a significant impact on the model predictions. Interactions between other variables had no significant impact on the model predictions. The ROC curves used to verify the predictability of the machine learning model showed an AUC of 0.750 (95% CI = 0.729–0.766), indicating good predictability (Figure 10).

Figure 8

Figure 8. SHAP summary plot. The plot ranked the factors by their importance, with liver metastasis being the most influential, followed by age, radiotherapy, stage, and other factors.

Figure 9

Figure 9. The multivariate dependence plot demonstrated the interactions between the different variables.

Figure 10

Figure 10. The ROC curve to verify the predictability of the machine learning model.

4 Discussion

In this study, we established a prognostic model for predicting the survival probability of patients with OM. The model incorporated 13 significant independent prognostic variables, including age, sex, tumor site, histologic subtype, stage, basal diameter, tumor thickness, liver metastasis, FMPI, marital status, and treatment with surgery/radiotherapy/chemotherapy. Previous studies have not consistently analyzed age and OS (5, 12). We used the X-tile software to determine the best cutoff value for age and found that patients aged ≥61 years had a worse prognosis. Our findings revealed that men had a shorter OS than women, corroborating the findings of previous studies (5, 13, 14). In a retrospective study of 8,033 patients, increased melanoma thickness was associated with an increased risk of metastasis. Each additional millimeter of thickness increased the risk of metastasis by 5% after 10 years (15). Furthermore, we included tumor thickness as a variable in our nomogram. Consistent with those of previous studies, our findings indicated that other factors related to the primary tumor, including tumor location, diameter, histological subtype, and distant metastasis, were independent prognostic factors (8, 16–18). In addition, our study identified FMPI and marital status as factors influencing survival. In our study on the treatment modalities affecting the survival prognosis of patients with OM, the risk of mortality was lower among patients who had undergone radiotherapy than among those who had undergone surgical intervention or chemotherapy. Compared with previous studies, this study identified more prognostic factors for predicting the survival of patients with OM. Because UM accounts for ~85% of all malignant OMs, most studies have established prediction models for UM (19). Unlike previous studies that targeted UM specifically, we established a survival prediction model for all patients with malignant OM, regardless of the specific site of the tumor. Given the rarity of OM, external validation of large amounts of data using this nomogram was challenging. Nevertheless, we conducted calibration and internal validation using multiple dimensions to evaluate the model prediction. The nomogram exhibited strong predictive performance for survival in patients with OM, with a C-index and AUC > 0.7. The use of the ROC curve, calibration curve, and DCA further validates its accuracy. The 10-fold cross-validation method provides a robust and reliable estimate of the model's performance (20). By repeatedly partitioning the dataset and evaluating the model on different subsets, we ensured consistent performance across different subsets of the data and were not overly dependent on a specific subset of the data (21). The AUC value of the 10-fold cross-ROC curve was also > 0.7, indicating good performance. This method is particularly effective in preventing overfitting and provides confidence that the model will generalize well to new, unseen data (22). In addition to the calibration and verification methods mentioned above, we introduced an innovative machine-learning algorithm for comparison with a nomogram.

In recent years, machine learning has emerged as a crucial tool for cancer prediction because it can learn from clinical data and construct accurate prediction models (23). However, the challenge with machine learning models lies in their black-box nature, which limits their application in clinical practice (24). XGBoost is an efficient gradient-boosting algorithm that has shown promising results in large-scale datasets (25, 26). SHAP, an interpretable machine-learning method, has been widely used to elucidate the prediction outcomes of complex models (25). SHAP assigns an importance weight to each feature, revealing its contribution to the model's prediction. The SHAP method combined with the XGBoost model revealed that liver metastasis, age, radiotherapy, and tumor stage had the most significant effects on the model. These variables were also included in the nomogram, demonstrating strong consistency between the machine prediction model and the nomogram. The SHAP value effectively captured the complex interaction effects and non-linear relationships (27). The ROC of our machine learning model showed good performance, with an AUC of 0.750. Additionally, the multivariate dependency graph highlighted that the interactions between age and liver metastasis, stage, and radiotherapy had the greatest impact on survival prediction.

We observed disparities in the most influential variables in both the nomogram and the machine learning models. The nomogram model identified the stage as the foremost determinant of survival prognosis, whereas the machine learning model prioritized liver metastasis as a pivotal variable. SHAP is rooted in the principles of game theory and provides a precise quantification of each feature's influence on the model output (27). Consequently, the significance of staging in the nomogram model may encompass the contribution of liver metastasis variables. Therefore, integrating conventional techniques with innovative machine learning approaches is crucial. Further exploration through deep learning analysis is warranted to achieve an optimal amalgamation of these methodologies for clinical applications.

Our study had certain limitations. The prediction model was developed based on the retrospective data, which inevitably introduced bias. Owing to database limitations, other variables that may impact survival prognosis, including Eastern Cooperative Oncology Group performance and new treatments such as targeted therapy and immunotherapy, were not collected. These variables should also be analyzed. The survival events in this study considered only OS, necessitating the inclusion of cancer-specific survival in future studies for comparative analysis. During the variable selection process, we acknowledged that brain and bone metastases may influence survival. However, the sample sizes for these variables were small, resulting in an unstable model. Incorporation of more patient data in further analyses is necessary to enhance the robustness of the model. Given that OM is a rare disease, accumulating a larger sample size and clinical information is crucial for reliable model training and external validation. Molecular pathological features, particularly genetic characteristics, can significantly affect the survival of patients with OM. Therefore, we aimed to combine clinicopathological features with genomic variables to construct a more accurate prognostic prediction model. Additionally, with advancements in artificial intelligence, these tools can be leveraged alongside traditional methods to develop an optimal model for predicting survival in rare diseases, such as OM.

5 Conclusion

In conclusion, we developed a nomogram incorporating 13 significant clinicopathological variables to predict survival in patients with OM. The synthesis of ROC and AUC values, calibration plots, DCA, 10-fold cross-validation, and machine learning models with SHAP values demonstrated the robust performance of the nomogram. These findings enhance our understanding of the prognosis of OM and assist clinicians in making informed decisions.

Data availability statement

The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found below: Data regarding the SEER program can be found on the SEER website (https://seer.cancer.gov).

Ethics statement

Because the study used a de-identified and publicly available database, ethical approval was not required. The design and analysis procedures were performed in accordance with the principles outlined in the Declaration of Helsinki. Additional information regarding the SEER program can be found on the SEER website (https://seer.cancer.gov). The studies were conducted in accordance with the local legislation and institutional requirements. The Ethics Committee/institutional review board waived the requirement of written informed consent for participation from the participants or the participants' legal guardians/next of kin.

Author contributions

MZ: Funding acquisition, Writing – original draft. MX: Data curation, Writing – original draft. MY: Funding acquisition, Writing – review & editing. ZH: Writing – review & editing.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This study was supported by a grant awarded to MZ and MY by Scientific Research Project of Putian University (#2021052, #2021060).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Culp MB, Benard V, Dowling NF, King J, Lu H, Rao C, et al. Ocular melanoma incidence rates and trends in the United States, 2001–2016. Eye. (2021) 35:687–9. doi: 10.1038/s41433-020-0885-1

PubMed Abstract | Crossref Full Text | Google Scholar

2. Wu M, Yavuzyigitoglu S, Brosens E, Ramdas WD, Kiliç E. Worldwide incidence of ocular melanoma and correlation with pigmentation-related risk factors. Invest Ophthalmol Vis Sci. (2023) 64:45. doi: 10.1167/iovs.64.13.45

PubMed Abstract | Crossref Full Text | Google Scholar

3. Arockiaraj BM, Cahoon EK, Sargen MR, Long E, Tucker MA, Mai JZ. Ambient ultraviolet radiation and ocular melanoma incidence in the United States, 2000–2019. Eye. (2024) 38:1–8. doi: 10.1038/s41433-024-02959-9

PubMed Abstract | Crossref Full Text | Google Scholar

4. van Poppelen NM, de Bruyn DP, Bicer T, Verdijk R, Naus N, Mensink H. Genetics of ocular melanoma: insights into genetics, inheritance and testing. Int J Mol Sci. (2021) 22:336. doi: 10.3390/ijms22010336

PubMed Abstract | Crossref Full Text | Google Scholar

5. Khoja L, Atenafu EG, Suciu S, Leyvraz S, Sato T, Marshall E. Meta-analysis in metastatic uveal melanoma to determine progression free and overall survival benchmarks: an international rare cancers initiative (IRCI) ocular melanoma study. Ann Oncol. (2019) 30:1370–80. doi: 10.1093/annonc/mdz176

PubMed Abstract | Crossref Full Text | Google Scholar

6. Cao X, Zeng J, Ou Y, Chen J, Lyu J, Zhou Q. A prognostic nomogram for the cancer-specific survival rate of choroidal melanoma using the Surveillance, Epidemiology, and End Results database. Front. Med. (2024) 11:1392336. doi: 10.3389/fmed.2024.1392336

PubMed Abstract | Crossref Full Text | Google Scholar

7. Wang Y, Xie M, Lin F, Sheng X, Zhao X, Zhu X, et al. Nomogram of uveal melanoma as prediction model of metastasis risk. Heliyon. (2023) 9:e18956. doi: 10.1016/j.heliyon.2023.e18956

PubMed Abstract | Crossref Full Text | Google Scholar

8. Mariani P, Dureau S, Savignoni A, Rouic LL, Levy-Gabriel C, Piperno-Neumann S, et al. Development of a prognostic nomogram for liver metastasis of uveal melanoma patients selected by liver MRI. Cancers. (2019) 11:863. doi: 10.3390/cancers11060863

PubMed Abstract | Crossref Full Text | Google Scholar

9. Dalvin LA, Zhang Q, Hamershock RA, Chang M, Yu MD, Mashayekhi A, et al. Nomogram for visual acuity outcome after iodine-125 plaque radiotherapy and prophylactic intravitreal bevacizumab for uveal melanoma in 1131 patients. Br J Ophthalmol. (2020) 104:697–702. doi: 10.1136/bjophthalmol-2019-314686

PubMed Abstract | Crossref Full Text | Google Scholar

10. Chen T, Guestrin C. “Xgboost: a scalable tree boosting system,” in Proceedings of the 22nd acm sigkdd International Conference on Knowledge Discovery and Data Mining, California, San Francisco (2016). p. 785–94.

Google Scholar

11. Lundberg SM, Lee S. A unified approach to interpreting model predictions. Adv Neural Inf Process Syst. (2017) 30:1–10. doi: 10.48550/arXiv.1705.07874

Crossref Full Text | Google Scholar

12. Rantala ES, Hernberg MM, Lundin M, Lundin J, Kivelä TT. Metastatic uveal melanoma managed with best supportive care. Acta Oncol. (2021) 60:135–9. doi: 10.1080/0284186X.2020.1817978

PubMed Abstract | Crossref Full Text | Google Scholar

13. Chi H, Peng G, Yang J, Zhang J, Song G, Xie X. Machine learning to construct sphingolipid metabolism genes signature to characterize the immune landscape and prognosis of patients with uveal melanoma. Front Endocrinol. (2022) 13:1056310. doi: 10.3389/fendo.2022.1056310

PubMed Abstract | Crossref Full Text | Google Scholar

14. Zhao M, Yu Y, Song Z. Identification and validation of a costimulatory molecule-related signature to predict the prognosis for uveal melanoma patients. Sci Rep. (2024) 14:9146. doi: 10.1038/s41598-024-59827-5

PubMed Abstract | Crossref Full Text | Google Scholar

15. Shields CL, Furuta M, Thangappan A, Nagori S, Mashayekhi A, Lally DR, et al. Metastasis of uveal melanoma millimeter-by-millimeter in 8033 consecutive eyes. Arch Ophthalmol (1960). (2009) 127:989. doi: 10.1001/archophthalmol.2009.208

PubMed Abstract | Crossref Full Text | Google Scholar

16. Shields CL, Kaliki S, Furuta M, Fulco E, Alarcon C, Shields JA. American Joint Committee on Cancer classification of uveal melanoma (anatomic stage) predicts prognosis in 7731 patients. Ophthalmol. (2015) 122:1180–6. doi: 10.1016/j.ophtha.2015.01.026

PubMed Abstract | Crossref Full Text | Google Scholar

17. Zeng Q, Yao Y, Zhao M. Development and validation of a nomogram to predict cancer-specific survival of uveal melanoma. BMC Ophthalmol. (2021) 21:230. doi: 10.1186/s12886-021-01968-6

PubMed Abstract | Crossref Full Text | Google Scholar

18. Liang X, Rong Y, Wang J, Zhang H. Primary sites of uveal melanoma asssociated with distinct survival outcomes and clinicopathological features: a SEER population-based study of 4359 cases. Int J Gen Med. (2021) 14:5221–32. doi: 10.2147/IJGM.S328910

PubMed Abstract | Crossref Full Text | Google Scholar

19. Chang AE, Karnell LH, Menck HR. The National Cancer Data Base report on cutaneous and noncutaneous melanoma: a summary of 84,836 cases from the past decade. Cancer: interdisciplinary Int J Am Cancer Soc. (1998) 83:1664–78. doi: 10.1002/(sici)1097-0142(19981015)83:8<1664::aid-cncr23>3.0.co;2-g

PubMed Abstract | Crossref Full Text | Google Scholar

20. Singh A, Melendez-Moreno A, Krohn J, Zabor EC. Predictive model for iris melanoma. Br J Ophthalmol. (2024) 12:1598–604. doi: 10.1136/bjo-2023-324558

PubMed Abstract | Crossref Full Text | Google Scholar

21. Shariati MM, Eslami S, Shoeibi N, Eslampoor A, Sedaghat M, Gharaei H, et al. Development, comparison, and internal validation of prediction models to determine the visual prognosis of patients with open globe injuries using machine learning approaches. BMC Med Inform Decis Mak. (2024) 24:131. doi: 10.1186/s12911-024-02520-4

PubMed Abstract | Crossref Full Text | Google Scholar

22. Cicinelli MV, Barlocci E, Giuffrè C, Rissotto F, Introini U, Bandello F. Integrating machine learning and traditional survival analysis to identify key predictors of foveal involvement in geographic atrophy. Invest Ohthalmol Vis Sci. (2024) 65:10. doi: 10.1167/iovs.65.5.10

PubMed Abstract | Crossref Full Text | Google Scholar

23. Ahmed TS, Shah J, Zhen YNB, Chua J, Wong DWK, Nusinovici S, et al. Ocular microvascular complications in diabetic retinopathy: Insights from machine learning. BMJ Open Diab Res Care. (2024) 12:e003758. doi: 10.1136/bmjdrc-2023-003758

PubMed Abstract | Crossref Full Text | Google Scholar

24. Ting DSW, Pasquale LR, Peng L, Campbell JP, Lee AY, Raman R, et al. Artificial intelligence and deep learning in ophthalmology. Br J Ophthalmol. (2019) 103:167–75. doi: 10.1136/bjophthalmol-2018-313173

PubMed Abstract | Crossref Full Text | Google Scholar

25. Park J, Kim J, Ryu D, Choi HY. Factors related to steroid treatment responsiveness in thyroid eye disease patients and application of SHAP for feature analysis with XGBoost. Front Endocrinol. (2023) 14:1079628. doi: 10.3389/fendo.2023.1079628

PubMed Abstract | Crossref Full Text | Google Scholar

26. Breeze F, Hossain RR, Mayo M, McKelvie J. Predicting ophthalmic clinic non-attendance using machine learning: development and validation of models using nationwide data. Clin Exper Ophthalmol. (2023) 51:764–74. doi: 10.1111/ceo.14310

PubMed Abstract | Crossref Full Text | Google Scholar

27. Sun JQ, Wu SN, Mou ZL, Wen JY, Wei H, Zou J, et al. Prediction model of ocular metastasis from primary liver cancer: machine learning-based development and interpretation study. Cancer Med. (2023) 12:20482–96. doi: 10.1002/cam4.6540

PubMed Abstract | Crossref Full Text | Google Scholar