A clinical prediction model for lung metastasis risk in osteosarcoma: A multicenter retrospective study

Background: Lung metastases (LM) have a poor prognosis of osteosarcoma. This study aimed to predict the risk of LM using the nomogram in patients with osteosarcoma.

Methods: A total of 1100 patients who were diagnosed as osteosarcoma between 2010 and 2019 in the Surveillance, Epidemiology and End Results (SEER) database were selected as the training cohort. Univariate and multivariate logistic regression analyses were used to identify independent prognostic factors of osteosarcoma lung metastases. 108 osteosarcoma patients from a multicentre dataset was as valiation data. The predictive power of the nomogram model was assessed by receiver operating characteristic curves (ROC) and calibration plots, and decision curve analysis (DCA) was utilized to interpret the accurate validity in clinical practice.

Results: A total of 1208 patients with osteosarcoma from both the SEER database(n=1100) and the multicentre database (n=108) were analyzed. Univariate and multivariate logistic regression analyses showed that Survival time, Sex, T-stage, N-stage, Surgery, Radiation, and Bone metastases were independent risk factors for lung metastasis. We combined these factors to construct a nomogram for estimating the risk of lung metastasis. Internal and external validation showed significant predictive differences (AUC 0.779, 0.792 respectively). Calibration plots showed good performance of the nomogram model.

Conclusions: In this study, a nomogram model for predicting the risk of lung metastases in osteosarcoma patients was constructed and turned out to be accurate and reliable through internal and external validation. Moreover we built a webpage calculator (https://drliwenle.shinyapps.io/OSLM/) taken into account nomogram model to help clinicians make more accurate and personalized predictions.

Introduction

Osteosarcoma is the most common malignant tumor of primary bone other than tumors of the lymphohematopoietic system. It is thought to originate from mesenchymal tissue, accounting for approximately 16% of primary bone tumors and 19% of malignant bone tumors, 90% of which are common osteosarcoma (1, 2). Some osteosarcoma patients were diagnosed with Lung metastasis (LM) at presentation, and about 50% of patients without LM at the beginning of the presentation were also developed LM during chemotherapy.

LM has a high mortality rate at an early stage and it can be mistaken for trauma or growing pains and neglected at the onset. In addition, severe side effects such as myelosuppression, gastrointestinal toxicity and hypersensitivity reactions and resistance to conventional chemotherapy greatly reduce the quality of life of patients (3, 4). Although the treatment model of osteosarcoma has been improved from “amputation-based” to “preoperative neoadjuvant chemotherapy, surgical resection of tumor, and postoperative adjuvant chemotherapy” in recent years, the 5-year survival rate of patients with LM <30% (5, 6). So there has been a demand for further study on osteosarcoma, especially the problem of LM to improve the quality of life of patients and overall survival rate (5, 6).

In recent years, nomogram have been extensively used worldwide to generate the likelihood of clinical events through complex computational formulas (7, 8). With the help of nomogram, clinicians can assess the risks of clinical events and then design individual treatment plans, determine the use of adjuvant therapies, optimize treatment protocols and consider appropriate patient counseling (9). Little research is found about LM combined with nomogram in osteosarcoma-related studies. Considering the important role of LM in the prognosis of osteosarcoma, this study aimed to use a nomogram to identify patients with osteosarcoma at high risk of LM.

Studies based on relevant Surveillance, Epidemiology and End Results (SEER) databases can target larger populations in different regions than single-centre studies. The SEER database (https://seer.cancer.gov/) currently collects approximately 34.6% of tumour incidence and survival data from US population-based cancer registries. It is considered the definitive source of tumour information in the US source (10). We extracted osteosarcoma patient data from SEER to build a prediction model, and further collected patient data from four medical centers in different regions of China for external validation of the model. In order to make the model truly clinically useful, we dynamized the model through a web calculator.

Methods

Data sources and inclusion criteria

The population data for this retrospective study included all patients diagnosed with osteosarcoma. We extracted 1100 patients diagnosed with osteosarcoma between 2010 and 2016 from the SEER database using the SEER*STAT (8.3.5) software,which divided into a training set and an internal validation set in the ratio of 7:3. The inclusion criteria were as follows:(1) ICD-O-3 for osteosarcoma:9180-9185;(2) complete clinical information with a clear survival or follow-up time;(3) age >18 years. Exclusion criteria were as follows: (1) clinical information was missing or unknown; (2)other primary tumour disease and cases with unknown metastatic status.

Demographic and clinical characteristics in the SEER database were as follows: Race, Age, Sex, Primary site, Histological differentiation, Laterality, Grade, T-stage, N-stage, Surgery, Radiation, Chemotherapy, Bone metastases,and lung metastases. We also recruited 108 patients from multi-centre data, as external validation.The Osteosarcoma’s tumor node metastasis (TNM) stage was evaluated based on the 7th edition of the American Joint Committee on Cancer (AJCC) staging manual. “SEER Combined Mets at DX-lung (2010 +)” was used to identify the presence of lung metastasis in a newly diagnosed.Two experts from each hospital were independently responsible for acquiring and processing the data to avoid subjective bias, and The data was reconfirmed by a third person.

Finally, we used a Microsoft Excel tool (Microsoft Excel, 2013, Redmond, USA) to record all data information accurately and sort it according to date.

Nomogram construction, validation and clinical application

A total of 1208 patients were identified by recording patient demographics and baseline clinical survey data, with 1770 patients as the training cohort, 330 patients an internal validation set from the SEER database and multicentre data as the validation dataset (n=108). Thereafter, all data were subjected to preliminary descriptive statistics to describe the baseline characteristics of the two cohorts. Baseline patient characteristics were compared between the training cohort and the validation cohort by chi-square test.

Firstly, we assessed variables predicting the occurrence of lung metastases in Osteosarcoma patients by univariate logistic regression analysis. Subsequently, multivariate logistic regression analysis was used to assess each variable at the 0.05 significance level and to identify independent factors associated with lung metastasis. Thirdly, the data for the training cohort was used to construct column plots based on these prognostic factors Univariate and multivariate logistic regression models were used to identify variables significantly associated with pulmonary metastases from Osteosarcoma in the SEER dataset. Columnar line graphs of lung metastasis prediction based on independent factors were then constructed from the SEER cohort. In addition, we performed decision curve analysis (DCA) to assess the clinical utility and value of the line graphs. After building the nomogram, we evaluated a series of metrics. For calibration, we first assessed the discriminatory ability of the new model using the area under the time-related subject operating characteristic (ROC) curve (AUC) to improve the accuracy and comprehensiveness of the comparison (11), with the higher area under the curve (AUC) being more accurate. All analyses were performed using R software(version 4.0.5). Two-tailed p-values less than 0.01 were considered statistically significant. The relationship between actual and predicted probabilities was verified by calibration curves. The agreement between the predicted probabilities and the actual situation using column line plots was assessed by plotting calibration plots (12). Finally, we used DCA to assess the clinical validity of the model (13). This study statement confirms that all methods were performed in accordance with relevant guidelines and regulations, and all experimental protocols were approved by the Ethics Committee of Xianyang Central Hospital, confirming that informed consent was obtained from all subjects or their legal guardians.

Statistical methods and software

In this study, normally distributed continuous variables such as Age and Survival time were expressed as mean ± standard deviation, while other categorical variables were expressed as numbers and percentages (N, %). SPSS statistical software (SPSS version 26.0, Chicago, USA) was used for the statistics and R software (version 4.0.5, http://www.Rproject.org) was used for the column line graphs development and evaluation. Firstly, univariate logistic regression analysis was used to identify risk factors associated with lung metastases, and patients with P<0.05 in univariate analysis were included in multivariate logistic regression analysis to identify independent prognostic factors. In addition, we calculated the corresponding 95% confidence intervals (95% CI). Based on the independent risk factors, column line plots predicting the development of lung metastases in patients with Osteosarcoma were constructed. Calibration plots were used to assess the accuracy of the columnar plots, while DCA curves were used to assess the clinical value of the columnar plots. Unlike sensitivity, specificity and area under the curve, DCA directly assesses the utility of clinical risk prediction models in decision making (14). p-values less than 0.05 were considered statistically significant.

Results

Baseline patient characteristics

A total of 1208 patients with Osteosarcoma were retrieved for further analysis based on inclusion criteria and screened for inclusion.1770 patients as the training cohort, 330 patients an internal validation set from the SEER database and 108 patients from the multicentre were included in the validation cohort. The training cohort was used to construct and internally validate the column line graphs, and the validation cohort was used for external validation. Detailed demographic information and clinical characteristics of the two cohorts and their comparisons are shown in Table 1. As shown in Table 1, there were no statistically significant differences between the SEER database and Multicenter cohorts in terms of Age, Survival time, Sex, Primary site, Differentiation, Grade, T stage, N stage, Surgery, Radiation, bone metastases and lung metastases (p>0.05), but there were significant differences in terms of race, laterality and chemotherapy (p< 0.05).

TABLE 1

Table 1 Baseline data table of the training group and the validation group.

In Table 2, out of a total of 1208 patients with Osteosarcoma, 220 developed pulmonary metastases and 988 did not. There were significant differences between the LM and NO-LM groups in terms of Survival time, Stage group, T stage, N stage,Surgery and Bone metastases (p<0.001). In the LM group, 135 (61.4%) patients underwent surgery compared to 844 (85.4%) in the NO-LM group. A total of 148 (12.3%) patients received Radiation and 1060 (87.7%) patients did not receive Radiation. A total of 969 (80.2%) patients received chemotherapy and 239 (19.8%) patients did not receive chemotherapy. A total of 1144 (94.7%) patients did not develop bone metastases, compared to 7 (0.6%) and 57 (4.7%) patients with unknown and developing bone metastases respectively. We also compared LM and NO-LM groups in the training cohort, most similar results were also found in Table S1.

TABLE 2

Table 2 Baseline data for patients presenting with and without lung metastases.

Univariate and multivariate logistic regression results

Risk variables for lung metastasis were analysed by applying univariate and multivariate logistic regression in patients with Osteosarcoma (Table 3). Univariate logistic models showed that significant factors such as Survival time, Sex, T stage, N stage, Surgery, Radiation, and Bone metastases were associated with lung metastasis. Further we used multivariate logistic regression analysis to confirm the association between T stage (T2, OR=2.337, 95% CI=1.517-3.601, p<0.001; T3, OR=4.197, 95% CI=1.727-10.199, p<0.01; Tx, OR=2.056, 95% CI=1.168-3.621, p< 0.05), N stage (NX, OR=2.110, 95% CI=1.133-3.928, p<0.05) and Bone metastases (yes, OR=4.329, 95% CI=2.265-8.275, p<0.001; unknown, OR=9.848, 95% CI=1.113-87.117, p<0.05) were independent risk factors, and Survival time (OR=0.973, 95% CI=0.963-0.983, p<0.001) and Surgery (yes, OR=0.540, 95% CI=0.355-0.822, p<0.01) were independent protective factors for lung metastases in patients with Osteosarcoma, and these factors were significantly associated with lung metastases in patients with Osteosarcoma.

TABLE 3

Table 3 Univariate and multivariate logistic regression analysis of risk factors for lung metastasis in patients with osteosarcoma.

Nomogram construction and verification

To predict the occurrence of lung metastasis in patients with Osteosarcoma, a nomogram was created based on the univariate and multivariate logistic regression results from the training set (Figure 1A). By counting each variable point, the total number of points may be correlated with the probability of lung metastasis in Osteosarcoma. As seen in Figure 1A, we found that bone metastases had the greatest effect on lung metastases, while surgery and sex having the least effect. The predicted nomogram was validated in both the SEER cohort and the multicentre cohort. The AUC is showed in Figure 2 A,and in internal validation and externalvalidation were 0.805 (95% CI: 0.758 to 0.846) and 0.753 (95% CI:0.661 to 0.831), respectively (Figures 2B, C). The calibration curve for the column line plot performed well in both the training and validation cohorts (Figures 1B–D). The results of the training and validation cohorts consistently showed that the predictive power of the nomogram was higher than that of the univariate factor.

FIGURE 1

Figure 1 Nomogram predicts LM in patients with osteosarcoma. The LM Nomogram (A) contains six prognostic factors and illustrates the risk of LM in patients by mapping their values to covariate scales. The calibration curves for the prediction training group (B), internal validation group (C) and multicentre validation group (D) are shown on the right.

FIGURE 2

Figure 2 ROC curve analysis of the predicted LM nomogram (A training group, B internal validation group and C multicentre validation group) for indicating the discriminatory ability of the nomogram. The specific AUC values were shown in Table 4.

In addition, we designed an online web-based calculator (https://drliwenle.shinyapps.io/OSLM/) to help us more effectively assess each patient’s risk of developing lung metastases, which can individually predict the probability of lung metastases based on the patient’s clinical characteristics. The results of the validation cohort showed that the new model significantly improves the accuracy and reliability of cancer prediction compared to the single factor in Table 4.

TABLE 4

Table 4 AUC of training group and validation group.

Clinical application of LM nomogram

Kaplan-Meier survival curves were plotted for overall survival in 1208 patients (Figures 3A, B). The results showed that Osteosarcoma patients with lung metastases had significantly lower survival levels compared to the NO-LM group both in all patients and external validation cohort (p<0.0001).

FIGURE 3

Figure 3 Kaplan-Meier survival curves were plotted for overall survival for patients with Osteosarcoma. A cohort of all patients was included to construct Kaplan-Meier survival curves (A). Kaplan-Meier survival curves in external validation cohort (B).

Meanwhile, we observed that the model has good clinical utility in predicting lung metastases in both the training cohort and the validation cohort of patients with Osteosarcoma. The net benefit of the training cohort was slightly higher than external validation cohort, which may be due to the limitation of the validation cohort size (Figure 4).

FIGURE 4

Figure 4 Nomogram determination curves (DCA) for lung metastasis risk. The red curve (number at high risk) indicates the number of people classified as positive (high risk) for each threshold probability for the Nomogram. The green curve (number at high risk with outcome) indicates the number of true positives at each threshold probability (A training group, B internal validation group and C multicentre validation group).

Discussion

Currently, despite the availability of treatments such as surgery and chemotherapy, the 5-year survival rate of osteosarcoma patients was still less than 70% (15), and metastases were statistically observed in 30-40% of osteosarcoma patients (16), of which the lung was the most common site of involvement (17, 18), and the 5-year survival rate for developing LM was even lower than 30% (19). Therefore, screening patients with osteosarcoma at high risk of LM has significant implications for guiding medical decisions, both for clinicians and patients.

There have been some previous studies on clinical prediction models for osteosarcoma, but the data for both the training and validation groups of these articles were obtained from the SEER database and were not externally validated. Meanwhile, these studies were all predictive of survival or distal metastasis and did not predict lung metastasis, the most critical factor affecting the survival of osteosarcoma patients. In terms of model application, these studies also only did nomogram and did not make it visual and dynamic (20–24). Nomogram is a quantitative tool to assess risk and benefit, and has made an outstanding contribution to modern medical decision making (25). Compared with conventional prognostic factor analysis, such as regression analysis and correlation analysis, nomogram provided a personalized estimate of the probability of an event by combining various important factors, providing a clear picture for clinicians to predict the outcome of a disease so that they can make decisions that were more valuable for treatment in the clinical process (26). Nomogram approaches have better applications and more advantages in the interpretation of complex mathematical models compared with traditional statistical presentation. The ability of a single influencing factor to predict the prognosis of patients is limited. Therefore, we need to combine all clinically relevant factors as much as possible to play an active role in accurately predicting lung metastases in patients with osteosarcoma. In addition, a web-based calculator has been designed to make it more applicable for clinical work.

Our study showed T stage, N stage and bone metastases were the most important risk variables for lung metastases, which is also consistent with previous findings (27). The Osteosarcoma presenting patients with metastatic lesions have a low survival rate compared to patients with focal OS (28). All we can conclude is that both regional staging and bone metastases are associated with a higher risk of mortality after controlling for confounding factors. Unlike conventional tumours, Osteosarcoma may not be sufficiently sensitive to respond to chemotherapy and radiotherapy (29), such that surgery plays an important part in the treatment of OS. In our study, both survival time and surgery were independent protective factors and negatively correlated with lung metastases and it was demonstrated that patients without surgery had a poor prognosis. In contrast, the extent of surgical resection is important for local control of the tumour (30). However, in our study, the SEER database did not contain detailed data on the extent of surgical resection, so we did not specifically investigate the impact of this variable

From the Nomogram plot, bone metastasis was the best predictor for the occurrence of LM in this study. According to previous literature, the probability of bone metastasis in osteosarcoma was only 8-10%, but the occurrence of bone metastasis usually implies combined multi-metastatic disease (27, 30, 31). This phenomenon may be related to the skeletal microenvironment. The skeletal microenvironment includes osteocytes, bone marrow endothelium, adipocytes and the immune environment, structures that participate in bone homeostasis in an as-yet-undefined manner by strictly regulating skeletal physiology in order to meet the various needs of the host. However, once the skeletal microenvironment was disrupted, it became a favorable “ground” for tumor metastasis (32). It had been shown that the development of lung adenocarcinoma was associated with the activity of osteoblasts and the accumulation of tumor-promoting sialic acid binding Ig-like lectin (Siglec) F high neutrophils in the lung. The depletion of osteoblasts in this experiment was found to reduce neutrophil accumulation significantly and tumor growth, revealing an important link between resident cell populations in bone and tumor growth at primary and distant metastatic sites (33) The above study suggested that bone metastases were indeed more likely to be combined with metastases from other sites of tumor. Furthermore, the results of logistic regression analysis of the current study confirmed this idea, with the risk of LM in osteosarcoma patients with the presence of bone metastases (OR = 9.868) being 9.8 times higher than that of those without detected bone metastases. Therefore, it was reasonable to assume that osteosarcoma patients with bone metastases present had a greater risk of developing LM.

In addition, the occurrence of LM was also associated with regional lymph node metastasis, and according to the results of logistic regression analysis, the risk of LM was significantly higher in both those with regional lymph node involvement and those with unknown regional lymph nodes which was 3-4 times higher than that of osteosarcoma patients without lymph node metastasis, based on osteosarcoma patients without lymph node metastasis. This view has been supported by relevant studies, and an autopsy study performed by Hatori et al. confirmed that LM through the lymphatic route were more common in multifocal osteosarcoma (34). However, there was no clear explanation for how lymphatic metastasis occurs in osteosarcoma (35). It has been found that osteosarcoma metastases with the destruction of the ctorex, and it was suspected that the metastatic route may occur through the lymphatic vessels of the synovial membrane and bursa (36). In terms of osteosarcoma lymphatic metastasis detection, one study detected lymph node metastasis in pediatric sarcoma patients by whole-body imaging with positron emission tomography using 18-fluorodeoxyglucose (FDG) as a tracer (FDG-PET). Moreover, this method was also effective for the detection of bone metastases. Finally, it was worth noting that this study concluded that gender was also a risk factor for LM. The results of the logistic regression analysis showed that the risk of LM was higher in men than in women. As shown by the OR value (0.664), the male-to-female ratio for LM was 1:0.664, which can be inferred that the risk of transfer was 1.5 times higher in men than in women.

Considering the tremendous impact of the presence of LM on the prognosis of patients with osteosarcoma, it was necessary to screen outpatients with osteosarcoma at high risk of LM. The advantage of the nomogram was that clinicians could calculate the risk of LM more accurately with a simple piece of paper without having to understand complex formulas. The DCA has also proven its clinical usefulness, and if clinicians focused on DCA, they can make better medical decisions to the patient.

Although our study has several strengths, it also has several limitations. Firstly, this study was limited by the data collection of retrospective studies, which may have inherent biases. For example, the specifics of tumour composition and different surgical approaches were not available, which may lead to selection bias and limit the need for prospective data collection for further analysis.Secondly,although tumour size is often an important clinical predictive indicator in tumor disease, it was not found to be an independent risk factor associated with outcome in our study. Finally, although our nomogram was developed and validated in the SEER database and multicentre databases, prospective external validation of the prediction models is still necessary. Considering that all factors required in the nomograms is available clinical data, it can be used to effectively individualise the predictions. Of course, we hope to continuously improve the model in the future by incorporating a variety of other clinical factors to better facilitate clinicians

Conclusions

In conclusion, our study was to develop a nomogram model on the prognosis of pulmonary metastases from osteosarcoma. We investigated a large number of patients based on the SEER database and a multicenter dataset. By analyzing the clinical characteristics and associated risk factors, we improve the prediction of lung metastasis risk and provide a basis for individualized treatment and follow-up strategies. The web-based calculator constructed in this study is an easy-to-use clinical tool that helps to promote personalized treatment.

Data availability statement

Publicly available datasets were analyzed in this study. This data can be found here: Training set data for the model are available from the SEER database (https://seer.cancer.gov/data-software/), and validation set data sets generated and/or analyzed in the current study can be obtained from the corresponding authors upon reasonable request.

Ethics statement

This study was approved by the Ethics Committee of Xianyang Central Hospital.

Author contributions

CLY, WLL, TTW, and GYL designed the study. SPZ, LHC and JMW performed the study and analyzed the data. SPZ and HSW wrote the manuscript. ZHH, WYL and CX provided the expert consultations and clinical suggestions. All authors contributed to the article and approved the submitted version.

Funding

This study was supported by: Shaanxi Provincial Health and Health Research Fund Project (2022E006).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fonc.2023.1001219/full#supplementary-material

Abbreviations

CI, Confidence interval; HR, Hazard ratio; LM, lung metastases; ROC, receiver operating characteristic; DCA, Decision curve analysis; CIC, clinical impact curves; SEER, Surveillance, Epidemiology, and End Results; AUC, area under the ROC curve.

References

1. Ritter J, Bielack SS. Osteosarcoma. Ann Oncol (2010) 21 (Suppl 7):vii320–325. doi: 10.1093/annonc/mdq276

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Casali PG, Bielack S, Abecassis N, Aro HT, Bauer S, Biagini R, et al. Bone sarcomas: ESMO-PaedCan-EURACAN clinical practice guidelines for diagnosis, treatment and follow-up. Ann Oncol (2018) 29(Suppl 4):iv79–95. doi: 10.1093/annonc/mdy310

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Shen G, Xing R, Zhang N, Chen C, Ma G, Yan X. Interfacial cohesion and assembly of bioadhesive molecules for design of long-term stable hydrophobic nanodrugs toward effective anticancer therapy. ACS Nano (2016) 10(6):5720–9. doi: 10.1021/acsnano.5b07276

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Zhang Y, Cai L, Li D, Lao YH, Liu D, Li M, et al. Tumor microenvironment-responsive hyaluronate-calcium carbonate hybrid nanoparticle enables effective chemotherapy for primary and advanced osteosarcomas. Nano Res (2018) 11:4806–22. doi: 10.1007/s12274-018-2066-0

CrossRef Full Text | Google Scholar

5. Guenther LM, Rowe RG, Acharya PT, Swenson DW, Meyer SC, Clinton CM, et al. Response evaluation criteria in solid tumors (RECIST) following neoadjuvant chemotherapy in osteosarcoma. Pediatr Blood Cancer (2018) 65(4). doi: 10.1002/pbc.26896

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Jaffe N. Historical perspective on the introduction and use of chemotherapy for the treatment of osteosarcoma. Adv Exp Med Biol (2014) 804:1–30. doi: 10.1007/978-3-319-04843-7_1

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Wang H, Zhang J, Bao S, Liu J, Hou F, Huang Y, et al. Preoperative MRI-based radiomic machine-learning nomogram may accurately distinguish between benign and malignant soft-tissue lesions: A two-center study. J Magnetic Resonance Imaging JMRI (2020) 52(3):873–82. doi: 10.1002/jmri.27111

CrossRef Full Text | Google Scholar

8. Zhou G, Xiao K, Gong G, Wu J, Zhang Y, Liu X, et al. A novel nomogram for predicting liver metastasis in patients with gastrointestinal stromal tumor: a SEER-based study. BMC Surg (2020) 20(1):298. doi: 10.1186/s12893-020-00969-4

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Khoo CY, Chai X, Quek R, Teo MCC, Goh BKP. Systematic review of current prognostication systems for primary gastrointestinal stromal tumors. Eur J Surg Oncol (2018) 44(4):388–94. doi: 10.1016/j.ejso.2017.12.006

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Siegel RL, Miller KD, Fuchs HE, Jemal A. Cancer statistics, 2021. CA: Cancer J Clin (2021) 71(1):7–33. doi: 10.3322/caac.21654

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Hilden J, Gerds TA. A note on the evaluation of novel biomarkers: do not rely on integrated discrimination improvement and net reclassification index. Stat Med (2014) 33(19):3405–14. doi: 10.1002/sim.5804

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Callegaro D, Miceli R, Bonvalot S, Ferguson P, Strauss DC, Levy A, et al. Development and external validation of two nomograms to predict overall survival and occurrence of distant metastases in adults after surgical resection of localised soft-tissue sarcomas of the extremities: a retrospective analysis. Lancet Oncol (2016) 17(5):671–80. doi: 10.1016/S1470-2045(16)00010-3

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decision Making (2006) 26(6):565–74. doi: 10.1177/0272989X06295361

CrossRef Full Text | Google Scholar

14. Van Calster B, Wynants L, Verbeek JFM, Verbakel JY, Christodoulou E, Vickers AJ, et al. Reporting and interpreting decision curve analysis: A guide for investigators. Eur Urol (2018) 74(6):796–804. doi: 10.1016/j.eururo.2018.08.038

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Harting MT, Blakely ML. Management of osteosarcoma pulmonary metastases. Semin Pediatr Surg (2006) 15(1):25–9. doi: 10.1053/j.sempedsurg.2005.11.005

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Duffaud F, Digue L, Mercier C, Dales JP, Baciuchka-Palmaro M, Volot F, et al. Recurrences following primary osteosarcoma in adolescents and adults previously treated with chemotherapy. Eur J Cancer (Oxford Engl 1990) (2003) 39(14):2050–7. doi: 10.1016/S0959-8049(03)00435-0

CrossRef Full Text | Google Scholar

17. Strauss SJ, McTiernan A, Whelan JS. Late relapse of osteosarcoma: implications for follow-up and screening. Pediatr Blood Cancer (2004) 43(6):692–7. doi: 10.1002/pbc.20154

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Ferrari S, Briccoli A, Mercuri M, Bertoni F, Cesari M, Longhi A, et al. Late relapse in osteosarcoma. J Pediatr Hematol/Oncol (2006) 28(7):418–22. doi: 10.1097/01.mph.0000212944.82361.1d

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Song K, Song J, Lin K, Chen F, Ma X, Jiang J, et al. Survival analysis of patients with metastatic osteosarcoma: a surveillance, epidemiology, and end results population-based study. Int Orthop (2019) 43(8):1983–91. doi: 10.1007/s00264-019-04348-4

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Zhou ZR, Wang WW, Li Y, Jin KR, Wang XY, Wang ZW, et al. In-depth mining of clinical data: the construction of clinical prediction model with r. Ann Trans Med (2019) 7(23):796. doi: 10.21037/atm.2019.08.63

CrossRef Full Text | Google Scholar

21. Chen B, Zeng Y, Liu B, Lu G, Xiang Z, Chen J, et al. Risk factors, prognostic factors, and nomograms for distant metastasis in patients with newly diagnosed osteosarcoma: A population-based study. Front Endocrinol (Lausanne) (2021) 12:672024. doi: 10.3389/fendo.2021.672024

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Lu S, Wang Y, Liu G, Wang L, Wu P, Li Y, et al. Construction and validation of nomogram to predict distant metastasis in osteosarcoma: a retrospective study. J Orthop Surg Res (2021) 16(1):231. doi: 10.1186/s13018-021-02376-8

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Ogura K, Fujiwara T, Yasunaga H, Matsui H, Jeon DG, Cho WH, et al. Development and external validation of nomograms predicting distant metastases and overall survival after neoadjuvant chemotherapy and surgery for patients with nonmetastatic osteosarcoma: A multi-institutional study. Cancer (2015) 121(21):3844–52. doi: 10.1002/cncr.29575

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Zhang J, Yang J, Wang HQ, Pan Z, Yan X, Hu C, et al. Development and validation of a nomogram for osteosarcoma-specific survival: A population-based study. Med (Baltimore) (2019) 98(23):e15988. doi: 10.1097/MD.0000000000015988

CrossRef Full Text | Google Scholar

25. Serenari M, Han KH, Ravaioli F, Kim SU, Cescon M. A novel nomogram based on liver stiffness to predict the comprehensive complication index after liver resection in patients with hepatocellular carcinoma. Digest Liver Dis (2020) 52(1):e17. doi: 10.1016/j.dld.2019.12.039

CrossRef Full Text | Google Scholar

26. Balachandran VP, Gonen M, Smith JJ, DeMatteo RP. Nomograms in oncology: more than meets the eye. Lancet Oncol (2015) 16(4):e173–180. doi: 10.1016/S1470-2045(14)71116-7

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Meazza C, Scanagatta P. Metastatic osteosarcoma: a challenging multidisciplinary treatment. Expert Rev Anticancer Ther (2016) 16(5):543–56. doi: 10.1586/14737140.2016.1168697

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Janeway KA, Barkauskas DA, Krailo MD, Meyers PA, Schwartz CL, Ebb DH, et al. Outcome for adolescent and young adult patients with osteosarcoma: a report from the children's oncology group. Cancer (2012) 118(18):4597–605. doi: 10.1002/cncr.27414

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Tsagozis P, Laitinen MK, Stevenson JD, Jeys LM, Abudu A, Parry MC. Treatment outcome of patients with chondroblastic osteosarcoma of the limbs and pelvis. Bone Joint J (2019) 101-b(6):739–44. doi: 10.1302/0301-620X.101B6.BJJ-2018-1090.R1

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Marina N, Gebhardt M, Teot L, Gorlick R. Biology and therapeutic advances for pediatric osteosarcoma. Oncologist (2004) 9(4):422–41. doi: 10.1634/theoncologist.9-4-422

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Shinkoda Y, Ijichi O, Tanabe T, Nishikawa T, Manago K, Ishikawa S, et al. Rapid progression of metastatic osteosarcoma after initiation of a reduced-intensity conditioning regimen with immunosuppressive fludarabine. Pediatr Transplant (2006) 10(7):822–5. doi: 10.1111/j.1399-3046.2006.00588.x

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Fornetti J, Welm AL, Stewart SA. Understanding the bone in cancer metastasis. J Bone Mineral Res (2018) 33(12):2099-113. doi: 10.1002/jbmr.3618

CrossRef Full Text | Google Scholar

33. Engblom C, Pfirschke C, Zilionis R, Da Silva Martins J, Bos SA, Courties G, et al. Osteoblasts remotely supply lung tumors with cancer-promoting SiglecF(high) neutrophils. Sci (New York NY) (2017) 358(6367):eaal5081. doi: 10.1126/science.aal5081

CrossRef Full Text | Google Scholar

34. Hatori M, Ohtani H, Yamada N, Uzuki M, Kokubun S. Synchronous multifocal osteosarcoma with lymphatic spread in the lung: An autopsy case report. Japanese J Clin Oncol (2001) 31(11):562–6. doi: 10.1093/jjco/hye118

CrossRef Full Text | Google Scholar

35. Dirik Y, Çınar A, Yumrukçal F, Eralp L. Popliteal lymph node metastasis of tibial osteoblastic osteosarcoma. Int J Surg Case Rep (2014) 5(11):840–4. doi: 10.1016/j.ijscr.2014.09.029

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Edwards JR, Williams K, Kindblom LG, Meis-Kindblom JM, Hogendoorn PC, Hughes D, et al. Lymphatics and bone. Hum Pathol (2008) 39(1):49–55. doi: 10.1016/j.humpath.2007.04.022

PubMed Abstract | CrossRef Full Text | Google Scholar