Radiomics Nomogram for Identifying Sub-1 cm Benign and Malignant Thyroid Lesions

Purpose: To develop and validate a radiomics nomogram for identifying sub-1 cm benign and malignant thyroid lesions.

Method: A total of 171 eligible patients with sub-1 cm thyroid lesions (56 benign and 115 malignant) who were treated in Yantai Yuhuangding Hospital between January and September 2019 were retrospectively collected and randomly divided into training (n = 136) and validation sets (n = 35). The radiomics features were extracted from unenhanced and arterial contrast-enhanced computed tomography images of each patient. In the training set, one-way analysis of variance and least absolute shrinkage and selection operator (LASSO) logistic regression were used to select the features related to benign and malignant lesions, and the LASSO algorithm was used to construct the radiomics signature. Combined with clinical independent predictive factors, a radiomics nomogram was constructed with a multivariate logistic regression model. The performance of the radiomics nomogram was evaluated by using the receiver operating characteristic (ROC) and calibration curves in the training and validation sets. The clinical usefulness was evaluated by using decision curve analysis (DCA).

Results: The radiomics signature consisting of 13 selected features achieved favorable prediction efficiency. The radiomics nomogram, which incorporated radiomics signature and clinical independent predictive factors including age and Thyroid Imaging Reporting and Data System category, showed good calibration and discrimination in the training (area under the ROC [AUC]: 0.853; 95% confidence interval [CI]: 0.797, 0.899) and validation sets (AUC: 0.851; 95% CI: 0.735, 0.931). DCA demonstrated that the nomogram was clinically useful.

Conclusion: As a noninvasive preoperative prediction tool, the radiomics nomogram incorporating radiomics signature and clinical predictive factors shows favorable predictive efficiency for identifying sub-1 cm benign and malignant thyroid lesions.

Introduction

According to ultrasound (US) screening and autopsy studies, thyroid lesions, which mainly include benign lesions and thyroid cancer, are common diseases with a prevalence of 30%–67% in the general population (1, 2). Papillary thyroid microcarcinoma (PTMC) is a subtype of papillary thyroid carcinoma (PTC), which is defined by the WHO as having a maximum diameter of 1.0 cm or less (3, 4). In recent decades, the incidence of thyroid cancer has rapidly increased throughout the world (5–7), with PTMC accounting for half of new cases (3, 8, 9). Although PTMCs usually have an indolent course, 24%–63% of patients may develop cervical regional lymph node metastasis at presentation (10, 11). To avoid overtreatment of thyroid lesions, benign lesions should be accurately distinguished from malignant ones before performing a biopsy or surgical resection (8, 9, 12).

At present, the main methods used to diagnose thyroid lesions are US and US-guided fine-needle aspiration biopsy (US-FNAB) (13, 14). However, US examinations show a diagnostic sensitivity of only 27%–63% for detecting lesion malignancy and are highly dependent on radiologists’ experience (15). Previous studies have shown that different radiologists can make different diagnoses after reviewing the US images of the same thyroid nodule (16). US-FNAB has a sensitivity of 54%–90% and a specificity of 60%–98% in diagnosing PTMC, and it has a sensitivity of approximately 30% in detecting non-diagnostic and indeterminate lesions (17–19). The American Thyroid Association guidelines do not recommend biopsy for sub-1 cm lesions that are highly suspicious for PTC on US. No non-invasive method can effectively and reliably diagnose PTMC. Thus, the methods for diagnosing sub-1 cm thyroid lesions should be improved, and the need for biopsy and diagnostic surgery should be reduced.

Computed tomography (CT), a common imaging examination method, is of great auxiliary value in preoperatively evaluating and determining the extent, localization, and lymph node status of the tumor (20). However, most diagnostic information from CT is based on visual inspection by a radiologist, who may miss critical diagnostic information. Thus, conventional CT is not effective in diagnosing thyroid lesions, especially sub-1 cm ones (21). In recent years, radiomics, which is the quantitative analysis of a large amount of data in medical images by means of computer technology, has received increasing attention due to its improved diagnosis and prediction accuracy (22–27). When combined with other relevant clinicopathological variables, radiomics-derived data can produce a more accurate and robust evidence-based decision system (28). Although the radiomics features of CT images can be used to help radiologists identify benign and malignant thyroid lesions (29), to the best of our knowledge, no radiomics-based study has predicted sub-1 cm benign and malignant thyroid lesions.

Therefore, the present study aimed to develop and validate a radiomics nomogram that incorporates radiomics features and clinical risk factors for identifying sub-1 cm benign and malignant thyroid lesions.

Materials and Methods

Patients

This study was approved by the ethics committee of the Yantai Yuhuangding Hospital. The informed consent requirement was waived. Patients with thyroid lesions who were treated at Yantai Yuhuangding Hospital from January to September 2019 were consecutively collected according to the inclusion and exclusion criteria. The inclusion criteria were as follows: (1) the pathology of surgical specimens was certain; (2) the maximum diameter of the thyroid lesion was ≤1 cm; and (3) clinical, US, and CT data were complete. The exclusion criteria were as follows: (1) biopsy or resection had been performed before the US and CT examination, (2) patients suffering from other tumor diseases, (3) patients with Hashimoto’s thyroiditis, and (4) cases with artifacts or noise affecting image quality. Figure 1 shows the recruitment pathway of patients. A total of 171 fully eligible patients with sub-1 cm thyroid lesions met the criteria (mean age, 46.47 ± 11.03 years; range, 21 to 71 years) were included. The patients were divided into two sets at a ratio of 8:2 using computer-generated random numbers: training set (n = 136; mean age, 46.21 ± 11.18 years; range, 21 to 71 years) and independent validation set (n = 35; mean age, 47.46 ± 10.54 years; range, 25 to 63 years).

FIGURE 1

Figure 1 Recruitment pathways for patients.

The clinical data of each patient were obtained by reviewing the medical records, including age, gender, Thyroid Imaging Reporting and Data System (TI-RADS) category, CT characteristics (maximum diameter, calcification, and location of nodule), free triiodothyronine (FT3), free thyroxine (FT4), and thyroid-stimulating hormone (TSH). Two senior radiologists reviewed all images and reassessed each lesion according to the 2017 American College of Radiology TI-RADS scoring criteria. The CT characteristics were re-examined and recorded by two radiologists with 10 years (Dr. A) and 8 years (Dr. B) of experience in the diagnosis of thyroid lesions. Any disagreements were resolved through negotiation to ensure accuracy and repeatability.

CT Image Acquisition

All patients underwent contrast-enhanced thyroid CT with a 64-slice spiral CT scanner (Siemens, Germany) or 256-slice spiral CT scanner (Philips, Netherlands). The exposure parameters for the CT scan were as follows: 120 kV, 300 effective mAs, scanning slice thickness 1.25 mm, pitch of 0.97, and matrix of 512 × 512. The scan range was from the skull base to the subclavian region. After unenhanced CT scanning, a contrast-enhanced CT scan was performed. Approximately 80–100 ml of nonionic contrast material (iopamidol, 320 mg/ml) was injected into the cubital vein at a rate of 3.5 ml/s, and then saline (30 ml) was injected at the same rate. Arterial-phase images were obtained at 30 s. All images were derived from the Picture Archiving and Communication System with the data format of Digital Imaging and Communications in Medicine. The images were imported into Radcloud (Huiying Medical Technology Co., Ltd.) and preprocessed. This process consisted of three steps, namely, standardization of the gray value of the region of interest (ROI), discretization of the gray level, and image resampling (30–32).

ROI Segmentation, Radiomics Feature Extraction, and Radiomics Signature Construction

Figures 2 and 3 present the radiomics workflow and study flowchart. The tumor ROI was manually segmented on the unenhanced and arterial contrast-enhanced CT images of the largest cross-sectional section. The manual segmentations were performed by Dr. A and Dr. B who were blinded to the pathologic results. Radiomics features (shape, firstorder, texture features) were extracted automatically from the ROIs of each image. ROI segmentation and radiomics feature extraction were performed using Radcloud (Huiying Medical Technology Co., Ltd.).

FIGURE 2

Figure 2 Flowchart showing the process of radiomics. An example of imaging segmentation and features extraction for patients with malignant nodule. ROI segmentation is performed on unenhanced and arterial contrast-enhanced computed tomography images. Features are extracted from the ROI, including tumor shape, intensity and texture. ROI, region of interest.

FIGURE 3

Figure 3 Study method flowchart.

Intra- and inter-class correlation coefficients (ICCs) were used to evaluate the intra- and inter-observer reproducibility of radiomics feature extraction. First, Dr. A and Dr. B randomly analyzed the images of 30 patients to evaluate the inter-class reproducibility. Two weeks later, Dr. A repeated the same procedure. An ICC greater than 0.8 indicates good agreement of the feature extraction. The remaining ROI segmentation was performed by Dr. A.

Then, one-way analysis of variance (ANOVA) and least absolute shrinkage and selection operator (LASSO) logistic regression were used to select the most useful predictive radiomics features from the training set. For the LASSO algorithm, the optimal penalization coefficient lambda (λ) was set by five-fold cross-validation, and radiomics features with non-zero coefficients within the training set were finally selected to construct the radiomics signature (33–35). The radiomics signature score (Rad-score) formula was generated using a linear combination of the selected features, which were weighted by their respective coefficients. Then, the Rad-score was calculated for each patient using this formula to compare the significant difference between the Rad-score of sub-1 cm benign and malignant lesions in the training and validation sets (Mann–Whitney U test). The predictive efficiency of the radiomics signature was quantified by using the area under the receiver operating characteristic (ROC) curve (AUC) in the training and validation sets.

Clinical Predictive Factors Selection and Radiomics Nomogram Construction

One-way ANOVA and multivariate logistic regression were performed to select independent predictive factors related to the identification of benign and malignant thyroid lesions, including clinical characteristics and Rad-score in the training set. Then, a radiomics nomogram was constructed on the basis of the multivariate logistic regression model.

Performance of the Radiomics Nomogram

ROC curves were plotted to assess the discrimination performance of the radiomics nomogram for sub-1 cm benign and malignant lesions. The calibration performance of the radiomics nomogram was evaluated by using calibration (agreement between the observations and the predicted malignant probability) curve. The main and ultimate purpose of using the nomogram is to combine the research results with clinical decisions so as to maximize patient benefit. However, discrimination and calibration could not capture the clinical consequences of a particular level of discrimination or degree of miscalibration. Therefore, decision curve analysis (DCA) was conducted to determine the clinical usefulness of the radiomics nomogram by quantifying the net benefits at different threshold probabilities in the validation set (net benefit is defined as true-positive rate minus false-positive rate, weighted by the relative harm of false-positive and false-negative results).

Statistical Analysis

All statistical analyses were performed using R software 3.5.3 and Python 2.7 software. Continuous data are reported as mean ± standard deviation or median (interquartile range). Categorical data are reported as numbers (%). All the levels of statistical significance were two-sided, and P-values < 0.05 were considered significant. The “SelectKBest” and “LassoCV” in Scikit-learn were used for selecting radiomics features. The “glm” function was used for multivariate logistic regression analysis. The “glmnet” package was used for LASSO logistic regression. The “vioplot” package was used to plot the violin diagram. The “Hmisc” package was used to draw the radiomics nomogram. The “pROC” package was used to plot the ROC curves and measure the AUCs. The “rms” package was used to plot the calibration curves. The “rmda” package was used to perform DCA.

Results

Clinical Characteristics

The clinical characteristics of patients in the training and validation sets are summarized in Table 1. No significant differences were found between the training and validation sets in terms of gender, TI-RADS category, CT characteristics (maximum diameter, calcification, and location of nodule), age, FT3, FT4, TSH, or pathology (P > 0.05).

TABLE 1

Table 1 Clinical characteristics of patients in the training and validation sets.

Construction of Radiomics Signature

In the training set, a total of 1409 radiomics features were extracted from each CT image. Favorable inter-observer and intra-observer reproducibility of feature extraction was achieved with intra-observer ICCs ranging from 0.856 to 0.914 and inter-observer ICCs ranging from 0.817 to 0.897. Then, 13 non-zero coefficient features associated with benign and malignant lesions were selected after using the ANOVA and LASSO algorithms (Figures 4A, B and Table 2), which included one morphological feature, six first-order features, and six texture features. Rad-score of each lesion was calculated by the 13 radiomics features. The results showed that the Rad-scores (median [interquartile range]) of the malignant lesions and benign lesions were significantly different (0.02 [-0.06 to 0.10] vs. -0.07 [-0.12 to 0.00], respectively, P < 0.05, Mann–Whitney U test); this difference was confirmed in the validation set (0.07 [-0.03, 0.15] vs. 0.03 [-0.08, 0.01], respectively, P < 0.05). The violin distribution of Rad-scores for benign and malignant lesions in the training and validation sets is presented in Figures 4C, D.

FIGURE 4

Figure 4 Computed tomography (CT) image features selection using one-way analysis of variance (ANOVA) and the least absolute shrinkage and selection operator (LASSO) logistic regression model in the training set. (A) The five-fold cross-validation and the minimal criteria process was used to generate the optimal penalization coefficient lambda (λ) in the LASSO model. The vertical line define the optimal values of λ, where the model provides its best fit to the data. The optimal λ value of 0.165 with -log (λ) =1.8 was selected. (B) LASSO coefficient profiles of the radiomics features. The vertical line was drawn at the value selected using five-fold cross-validation, where optimal λ resulted in 13 nonzero coefficients. Violin distribution of Rad-score for benign and malignant nodules in the training (C) and Validation (D) Sets. Green violin plots show data distribution of benign nodules, blue ones data distribution of malignant nodules. In each violin plot, the white point represents the median value of each group; the vertical black line represents the range.

TABLE 2

Table 2 Radiomics features selected in ANOVA and LASSO regression analysis.

Construction of Radiomics Nomogram

After performing one-way ANOVA and multivariate logistic regression, age, TI-RADS category, and Rad-score were identified as final predictors of sub-1 cm thyroid malignancy. A radiomics nomogram incorporating these three predictors was constructed (Figure 5).

FIGURE 5

Figure 5 Radiomics nomogram for the prediction of benign and malignant thyroid nodules of sub-1cm.

Performance of Radiomics Nomogram

Figures 6A, B show the ROC curves of the nomogram, Rad-score, and clinical prediction model in the training and validation sets. The results of AUCs for the nomogram, Rad-score, and clinical prediction model were 0.853 (95% confidence interval [CI]: 0.797, 0.899), 0.742 (95% CI: 0.676, 0.801), and 0.813 (95% CI: 0.752, 0.864) in the training set and 0.851 (95% CI: 0.735, 0.931), 0.707 (95% CI: 0.574, 0.818), and 0.775 (95% CI: 0.648, 0.873) in the validation set, respectively. The sensitivity and specificity of three models in the training and validation sets were exhibited in the Table 3, which showed the radiomics nomogram had good discrimination efficiency. Figure 6C illustrates the calibration curve of the radiomics nomogram. The calibration curve showed good calibration in the training set. The favorable calibration of the radiomics nomogram was confirmed with the validation set (Figure 6D). DCA was used to assess the clinical usefulness of the radiomics nomogram, radiomics signature, and clinical prediction model in the validation set (Figure 7). If the threshold probability of clinical decision was between 0.0 and 1.0,using the nomogram to predict malignancy provided more benefit than either the treat-all (assuming all lesions were malignant) or treat-none strategy (assuming all lesions were benign). Moreover, the use of radiomics nomograms to predict malignancy provided more net benefit than the use of the radiomics signature alone or clinical prediction model alone.

FIGURE 6

Figure 6 Receiver operating characteristic (ROC) curves of the nomogram (red lines), Rad-score model (brown lines) and clinical model (blue lines) in the training (A) and validation (B) sets, respectively. Calibration curves of the nomogram in the training (C) and validation (D) sets, respectively. The diagonal dotted line represents an ideal prediction, while the solid lines represent the performance of the nomogram. Closer fit to the diagonal dotted line indicates a better prediction.

TABLE 3

Table 3 Predictive performance of three models.

FIGURE 7

Figure 7 Decision curve analysis (DCA) of each model derived from the validation set. The y-axis measures the net benefit. The x-axis shows the corresponding risk threshold. The grey line represents the assumption that all lesions were malignant. The black line represents the assumption that all nodules were benign. If the threshold probability was more than 40%, using the nomogram to predict malignancy added more benefit than the Rad-score model (red line) and clinical model (blue line).

Discussion

In recent years, the morbidity of PTMC has dramatically increased. Studies have shown that some PTMCs can be associated with highly aggressive histological variants and even exhibit early localized invasion or lymph node and distant metastasis (36–39). Unfortunately, the accuracy of diagnosing PTMC is inefficient, resulting in a proportion of patients being mistreated or misdiagnosed. In the present study, the potential ability of CT-based radiomics for identifying sub-1 cm benign and malignant thyroid lesions was discussed. Our results indicated that the radiomics nomogram combined with radiomics signature and clinical risk factors could preoperatively predict small thyroid lesions with good performance.

To construct the radiomics signature, a LASSO logistic regression model was used to reduce the radiomics features. This method is widely used in discriminating benign and malignant lesions (37, 40, 41), and it is designed to avoid overfitting (42, 43). In our study, 13 radiomics features were finally selected as the most closely related features to the sub-1 cm thyroid lesion status, including 1 shape feature, 6 first order statistics features, 2 gray level dependence matrix (GLDM)-derived texture features, 2 gray level run-length matrix (GLRLM)-derived texture features, and 2 gray level size zone matrix (GLSZM)-derived texture features. Among them, sphericity accounted for the greatest weighted, indicating that the shape feature of the lesion may be the most important feature affecting the diagnosis of sub-1 cm benign and malignant thyroid lesions. Several studies have shown that shape features differentiate benign and malignant lesions on the basis of CT scans (44–46). At the same time, sphericity was inversely correlated with the radiomics signature, which is consistent with the findings of Limkin et al. (47). The radiomics signature based on unenhanced and arterial contrast-enhanced CT images showed good discrimination ability in the training (AUC: 0.742) and validation sets (AUC: 0.707).

It is well known that US and US-FNAB have significant advantages in determining thyroid lesions, but Zhang et al. (48) found that the sensitivity, specificity, and AUC of US in identifying PTMC were only 0.684, 0.771 and 0.728, which means that the diagnostic ability of US in PTMC is varies greatly. On the other hand, US-FNAB did show a greater advantage in the diagnosis of PTMC, Gao et al. (49) showed that the sensitivity, specificity, and AUC of US-FNAB in identifying PTMC were 0.988, 0.905, and 0.947, respectively. As a contrast, in our study, the sensitivity, specificity, and AUC of the nomogram model were 0.775, 0.790, and 0.851, respectively. However, CT is a more objective and non-invasive option.

The present study indicated that age and TI-RADS category were significant predictive factors. However, whether age is one of the important clinical risk factors of PTC still remains to be elucidated. Chen et al. (50) recently provided evidence that age, margin, shape, echogenic foci, echogenicity, and nodule halo sign are independent risk factors, whereas Liang et al. (51) reported that age has no significant relevance with PTC diagnosis.

The present study has several strengths. First, an independent validation set was used to verify the discrimination ability of the nomogram model. The results also had good diagnostic ability (AUC: 0.851), which demonstrates that the nomogram model has good generalization ability. However, the previous study failed to determine the usefulness of the radiomics nomogram in the clinical setting (52, 53). Hence, DCA was used to assess whether the radiomics nomogram could improve individual benefit. The results showed that if the threshold probability was more than 0, the predictive ability of the radiomics nomogram was more favorable than that with or without patient treatment. To improve the feature recognition rate, the gray value of ROI was standardized, discretized, and resampled. It effectively improved the repeatability of the research results. Finally, to comprehensively reflect the radiomics features of PTMC, unenhanced and arterial contrast-enhanced CT images were extracted for radiomics analysis.

This study has several limitations. First, bias is inevitable as the present study is retrospective in nature. Prospective studies are needed to control for confounding variables. Second, this study utilized a single center and had a small sample size. A large sample size and multiple centers are needed to improve the efficiency of the model. Third, a manual method is applied to image segmentation. Although manual segmentation is the gold standard, it may increase the variability of feature extraction. To avoid this disadvantage, the consistency of feature extraction was validated by using ICC.

In conclusion, this study presents a noninvasive predictive tool that incorporates CT radiomics signature and clinical risk factors. The radiomics nomogram shows favorable predictive accuracy in identifying sub-1 cm benign and malignant thyroid lesions. Multicenter retrospective validation and prospective randomized clinical trials should be performed in subsequent studies to obtain high-level evidence for the clinical application of this nomogram.

Data Availability Statement

The original contributions presented in the study are included in the article/supplementary material. Further inquiries can be directed to the corresponding authors.

Ethics Statement

This study was approved by the ethics committee of the Yantai Yuhuangding Hospital.

Author Contributions

XW and JL implemented the literature searching, and manuscript writing. YM implemented data collecting, ROI segmentation literature searching, and manuscript writing. YY contributed to data analysis and figures making. JC implemented the algorithm and software development. NM identified the radiological characteristics of PTMC, and estimated and adjusted the accuracy of ROIs. XS conducted the design, quality control, and data interpretation of this study. All authors contributed to the article and approved the submitted version.

Funding

This study was supported by the “Taishan Scholar” Project (No. ts20190991).

Conflict of Interest

JC was employed by Huiying Medical Technology Co., Ltd.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

1. Mortensen JD, Woolner LB, Bennett WA. Gross and Microscopic Findings in Clinically Normal Thyroid Glands. J Clin Endocrinol Metab (1955) 15:1270–80. doi: 10.1210/jcem-15-10-1270

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Gharib H, Papini E, Paschke R, Duick DS, Valcavi R, Hegedus L, et al. American Association of Clinical Endocrinologists, Associazione Medici Endocrinologi, and European Thyroid Association Medical Guidelines for Clinical Practice for the Diagnosis and Management of Thyroid Nodules: Executive Summary of Recommendations. J Endocrinol Invest (2010) 33:51–6. doi: 10.1007/BF03346587

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Brito JP, Al Nofal A, Montori VM, Hay ID, Morris JC. The Impact of Subclinical Disease and Mechanism of Detection on the Rise in Thyroid Cancer Incidence: A Population-Based Study in Olmsted County, Minnesota During 1935 Through 2012. Thyroid Off J Am Thyroid Assoc (2015) 25:999–1007. doi: 10.1089/thy.2014.0594

CrossRef Full Text | Google Scholar

4. Sobin LH. Histological Typing of Thyroid Tumours. Histopathology (1990) 16:513. doi: 10.1111/j.1365-2559.1990.tb01559.x

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Siegel RL, Miller KD, Jemal A. Cancer Statistics, 2019. CA: Cancer J Clin (2019) 69:7–34. doi: 10.3322/caac.21551

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Bann DV, Goyal N, Camacho F, Goldenberg D. Increasing Incidence of Thyroid Cancer in the Commonwealth of Pennsylvania. JAMA Otolaryngol Head Neck Surg (2014) 140:1149–56. doi: 10.1001/jamaoto.2014.1709

PubMed Abstract | CrossRef Full Text | Google Scholar

7. La Vecchia C, Malvezzi M, Bosetti C, Garavello W, Bertuccio P, Levi F, et al. Thyroid Cancer Mortality and Incidence: A Global Overview. Int J Cancer (2015) 136:2187–95. doi: 10.1002/ijc.29251

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Noguchi S, Yamashita H, Uchino S, Watanabe S. Papillary Microcarcinoma. World J Surg (2008) 32:747–53. doi: 10.1007/s00268-007-9453-0

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Qu H, Sun GR, Liu Y, He QS. Clinical Risk Factors for Central Lymph Node Metastasis in Papillary Thyroid Carcinoma: A Systematic Review and Meta-Analysis. Clin Endocrinol (2015) 83:124–32. doi: 10.1111/cen.12583

CrossRef Full Text | Google Scholar

10. Zhang L, Wei WJ, Ji QH, Zhu YX, Wang ZY, Wang Y, et al. Risk Factors for Neck Nodal Metastasis in Papillary Thyroid Microcarcinoma: A Study of 1066 Patients. J Clin Endocrinol Metab (2012) 97:1250–7. doi: 10.1210/jc.2011-1546

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Adam MA, Pura J, Goffredo P, Dinan MA, Reed SD, Scheri RP, et al. Presence and Number of Lymph Node Metastases are Associated With Compromised Survival for Patients Younger Than Age 45 Years With Papillary Thyroid Cancer. J Clin Oncol Off J Am Soc Clin Oncol (2015) 33:2370–5. doi: 10.1200/jco.2014.59.8391

CrossRef Full Text | Google Scholar

12. Luo Y, Zhao Y, Chen K, Shen J, Shi J, Lu S, et al. Clinical Analysis of Cervical Lymph Node Metastasis Risk Factors in Patients With Papillary Thyroid Microcarcinoma. J Endocrinol Invest (2019) 42:227–36. doi: 10.1007/s40618-018-0908-y

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Shi RY, Yao QY, Zhou QY, Lu Q, Suo ST, Chen J, et al. Preliminary Study of Diffusion Kurtosis Imaging in Thyroid Nodules and its Histopathologic Correlation. Eur Radiol (2017) 27:4710–20. doi: 10.1007/s00330-017-4874-0

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Raue F, Frank-Raue K. Thyroid Cancer: Risk-Stratified Management and Individualized Therapy. Clin Cancer Res (2016) 22:5012–21. doi: 10.1158/1078-0432.ccr-16-0484

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Remonti LR, Kramer CK, Leitao CB, Pinto LC, Gross JL. Thyroid Ultrasound Features and Risk of Carcinoma: A Systematic Review and Meta-Analysis of Observational Studies. Thyroid Off J Am Thyroid Assoc (2015) 25:538–50. doi: 10.1089/thy.2014.0353

CrossRef Full Text | Google Scholar

16. Huang BL, Ebner SA, Makkar JS, Bentley-Hibbert S, McConnell RJ, Lee JA, et al. A Multidisciplinary Head-to-Head Comparison of American College of Radiology Thyroid Imaging and Reporting Data System and American Thyroid Association Ultrasound Risk Stratification Systems. Oncol (2019) 25:398–403. doi: 10.1634/theoncologist.2019-0362

CrossRef Full Text | Google Scholar

17. Hang JF, Hsu CY, Lai CR. Thyroid Fine-Needle Aspiration in Taiwan: The History and Current Practice. J Pathol Trans Med (2017) 51:560–4. doi: 10.4132/jptm.2017.09.20

CrossRef Full Text | Google Scholar

18. Magri F, Chytiris S, Zerbini F, Capelli V, Gaiti M, Carbone A, et al. Maximal Stiffness Evaluation by Real-Time Ultrasound Elastography, an Improved Tool for the Differential Diagnosis of Thyroid Nodules. Endocrine Pract Off J Am Coll Endocrinol Am Assoc Clin Endocrinol (2015) 21:474–81. doi: 10.4158/ep14504.or

CrossRef Full Text | Google Scholar

19. Eszlinger M, Ullmann M, Ruschenburg I, Bohme K, Gorke F, Franzius C, et al. Low Malignancy Rates in Fine-Needle Aspiration Cytologies in a Primary Care Setting in Germany. Thyroid Off J Am Thyroid Assoc (2017) 27:1385–92. doi: 10.1089/thy.2017.0167

CrossRef Full Text | Google Scholar

20. Wu S, Zheng J, Li Y, Yu H, Shi S, Xie W, et al. A Radiomics Nomogram for the Preoperative Prediction of Lymph Node Metastasis in Bladder Cancer. Clin Cancer Res An Off J Am Assoc Cancer Res (2017) 23:6904–11. doi: 10.1158/1078-0432.ccr-17-1510

CrossRef Full Text | Google Scholar

21. Gao M, Ge M, Ji Q, Cheng R, Lu H, Guan H, et al. Chinese Expert Consensus and Guidelines for the Diagnosis and Treatment of Papillary Thyroid Microcarcinoma. Cancer Biol Med (2017) 14:203–11. doi: 10.20892/j.issn.2095-3941.2017.0051

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Jiang M, Li C, Tang S, Lv W, Yi A, Wang B, et al. Nomogram Based on Shear-Wave Elastography Radiomics can Improve Preoperative Cervical Lymph Node Staging for Papillary Thyroid Carcinoma. Thyroid Off J Am Thyroid Assoc (2020) 30:885–97. doi: 10.1089/thy.2019.0780

CrossRef Full Text | Google Scholar

23. Bi WL, Hosny A, Schabath MB, Giger ML, Birkbak NJ, Mehrtash A, et al. Artificial Intelligence in Cancer Imaging: Clinical Challenges and Applications. CA: Cancer J Clin (2019) 69:127–57. doi: 10.3322/caac.21552

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Khorrami M, Prasanna P, Gupta A, Patil P, Velu PD, Thawani R, et al. Changes in CT Radiomic Features Associated With Lymphocyte Distribution Predict Overall Survival and Response to Immunotherapy in Non-Small Cell Lung Cancer. Cancer Immunol Res (2020) 8:108–19. doi: 10.1158/2326-6066.cir-19-0476

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Liu Z, Li Z, Qu J, Zhang R, Zhou X, Li L, et al. Radiomics of Multiparametric MRI for Pretreatment Prediction of Pathologic Complete Response to Neoadjuvant Chemotherapy in Breast Cancer: A Multicenter Study. Clin Cancer Res An Off J Am Assoc Cancer Res (2019) 25:3538–47. doi: 10.1158/1078-0432.ccr-18-3190

CrossRef Full Text | Google Scholar

26. Scialpi M, Bianconi F, Cantisani V, Palumbo B. Radiomic Machine Learning: Is it Really a Useful Method for the Characterization of Prostate Cancer? Radiology (2019) 291:269–70. doi: 10.1148/radiol.2019182940

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Kim JY, Park JE, Jo Y, Shim WH, Nam SJ, Kim JH, et al. Incorporating Diffusion- and Perfusion-Weighted MRI Into a Radiomics Model Improves Diagnostic Performance for Pseudoprogression in Glioblastoma Patients. Neuro-oncology (2019) 21:404–14. doi: 10.1093/neuonc/noy133

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Huang YQ, Liang CH, He L, Tian J, Liang CS, Chen X, et al. Development and Validation of a Radiomics Nomogram for Preoperative Prediction of Lymph Node Metastasis in Colorectal Cancer. J Clin Oncol Off J Am Soc Clin Oncol (2016) 34:2157–64. doi: 10.1200/jco.2015.65.9128

CrossRef Full Text | Google Scholar

29. Peng W, Liu C, Xia S, Shao D, Chen Y, Liu R, et al. Thyroid Nodule Recognition in Computed Tomography Using First Order Statistics. Biomed Eng Online (2017) 16:67. doi: 10.1186/s12938-017-0367-2

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Zwanenburg A, Vallières M, Abdalah MA, Aerts H, Andrearczyk V, Apte A, et al. The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-Based Phenotyping. Radiology (2020) 295:328–38. doi: 10.1148/radiol.2020191145

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Shafiq-Ul-Hassan M, Zhang GG, Latifi K, Ullah G, Hunt DC, Balagurunathan Y, et al. Intrinsic Dependencies of CT Radiomic Features on Voxel Size and Number of Gray Levels. Med Physics (2017) 44:1050–62. doi: 10.1002/mp.12123

CrossRef Full Text | Google Scholar

32. Larue R, van Timmeren JE, de Jong EEC, Feliciani G, Leijenaar RTH, Schreurs WMJ, et al. Influence of Gray Level Discretization on Radiomic Feature Stability for Different CT Scanners, Tube Currents and Slice Thicknesses: A Comprehensive Phantom Study. Acta Oncol (Stockholm Sweden) (2017) 56:1544–53. doi: 10.1080/0284186x.2017.1351624

CrossRef Full Text | Google Scholar

33. Limkin EJ, Sun R, Dercle L, Zacharaki EI, Robert C, Reuze S, et al. Promises and Challenges for the Implementation of Computational Medical Imaging (Radiomics) in Oncology. Ann Oncol Off J Eur Soc Med Oncol (2017) 28:1191–206. doi: 10.1093/annonc/mdx034

CrossRef Full Text | Google Scholar

34. Gillies RJ, Kinahan PE, Hricak H. Radiomics: Images are More Than Pictures, They Are Data. Radiology (2016) 278:563–77. doi: 10.1148/radiol.2015151169

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Fetit AE, Novak J, Rodriguez D, Auer DP, Clark CA, Grundy RG, et al. Radiomics in Paediatric Neuro-Oncology: A Multicentre Study on MRI Texture Analysis. NMR Biomed (2018) 31. doi: 10.1002/nbm.3781

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Nam-Goong IS, Kim HY, Gong G, Lee HK, Hong SJ, Kim WB, et al. Ultrasonography-Guided Fine-Needle Aspiration of Thyroid Incidentaloma: Correlation With Pathological Findings. Clin Endocrinol (2004) 60:21–8. doi: 10.1046/j.1365-2265.2003.01912.x

CrossRef Full Text | Google Scholar

37. Liu A, Wang Z, Yang Y, Wang J, Dai X, Wang L, et al. Preoperative Diagnosis of Malignant Pulmonary Nodules in Lung Cancer Screening With a Radiomics Nomogram. Cancer Commun (London England) (2020) 40:16–24. doi: 10.1002/cac2.12002

CrossRef Full Text | Google Scholar

38. Yang L, Shen W, Sakamoto N. Population-Based Study Evaluating and Predicting the Probability of Death Resulting From Thyroid Cancer and Other Causes Among Patients With Thyroid Cancer. J Clin Oncol Off J Am Soc Clin Oncol (2013) 31:468–74. doi: 10.1200/jco.2012.42.4457

CrossRef Full Text | Google Scholar

39. Wang K, Xu J, Li S, Liu S, Zhang L. Population-Based Study Evaluating and Predicting the Probability of Death Resulting From Thyroid Cancer Among Patients With Papillary Thyroid Microcarcinoma. Cancer Med (2019) 8:6977–85. doi: 10.1002/cam4.2597

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Wang H, Zhang J, Bao S, Liu J, Hou F, Huang Y, et al. Preoperative MRI-Based Radiomic Machine-Learning Nomogram May Accurately Distinguish Between Benign and Malignant Soft-Tissue Lesions: A Two-Center Study. J Magnetic Resonance Imaging JMRI (2020) 52:873–82. doi: 10.1002/jmri.27111

CrossRef Full Text | Google Scholar

41. Zhang Q, Peng Y, Liu W, Bai J, Zheng J, Yang X, et al. Radiomics Based on Multimodal MRI for the Differential Diagnosis of Benign and Malignant Breast Lesions. J Magnetic Resonance Imaging JMRI (2020) 52:596–607. doi: 10.1002/jmri.27098

CrossRef Full Text | Google Scholar

42. Gui J, Li H. Penalized Cox Regression Analysis in the High-Dimensional and Low-Sample Size Settings, With Applications to Microarray Gene Expression Data. Bioinf (Oxford England) (2005) 21:3001–8. doi: 10.1093/bioinformatics/bti422

CrossRef Full Text | Google Scholar

43. Hepp T, Schmid M, Gefeller O, Waldmann E, Mayr A. Approaches to Regularized Regression - A Comparison Between Gradient Boosting and the Lasso. Methods Inf Med (2016) 55:422–30. doi: 10.3414/me16-01-0033

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Jun W, Xia L, Di D, Jiangdian S, Min X, Yali Z, et al. Prediction of Malignant and Benign of Lung Tumor Using a Quantitative Radiomic Method. Conf Proc Annu Int Conf IEEE Eng Med Biol Soc IEEE Eng Med Biol Soc Annu Conference (2016) 2016:1272–5. doi: 10.1109/embc.2016.7590938

CrossRef Full Text | Google Scholar

45. Pena E, Ojiaku M, Inacio JR, Gupta A, Macdonald DB, Shabana W, et al. Can CT and MR Shape and Textural Features Differentiate Benign Versus Malignant Pleural Lesions? Acad Radiol (2017) 24:1277–87. doi: 10.1016/j.acra.2017.03.006

PubMed Abstract | CrossRef Full Text | Google Scholar

46. He X, Sahiner B, Gallas BD, Chen W, Petrick N. Computerized Characterization of Lung Nodule Subtlety Using Thoracic CT Images. Phys Med Biol (2014) 59:897–910. doi: 10.1088/0031-9155/59/4/897

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Limkin EJ, Reuze S, Carre A, Sun R, Schernberg A, Alexis A, et al. The Complexity of Tumor Shape, Spiculatedness, Correlates With Tumor Radiomic Shape Features. Sci Rep (2019) 9:4329. doi: 10.1038/s41598-019-40437-5

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Zhang Y, Pan J, Xu D, Yang Z, Sun J, Sun L, et al. Combination of Serum microRNAs and Ultrasound Profile as Predictive Biomarkers of Diagnosis and Prognosis for Papillary Thyroid Microcarcinoma. Oncol Rep (2018) 40:3611–24. doi: 10.3892/or.2018.6776

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Gao L, Ma B, Zhou L, Wang Y, Yang S, Qu N, et al. The Impact of Presence of Hashimoto’s Thyroiditis on Diagnostic Accuracy of Ultrasound-Guided Fine-Needle Aspiration Biopsy in Subcentimeter Thyroid Nodules: A Retrospective Study From FUSCC. Cancer Med (2017) 6:1014–22. doi: 10.1002/cam4.997

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Chen L, Zhang J, Meng L, Lai Y, Huang W. A New Ultrasound Nomogram for Differentiating Benign and Malignant Thyroid Nodules. Clin Endocrinol (2019) 90:351–9. doi: 10.1111/cen.13898

CrossRef Full Text | Google Scholar

51. Liang J, Huang X, Hu H, Liu Y, Zhou Q, Cao Q, et al. Predicting Malignancy in Thyroid Nodules: Radiomics Score Versus 2017 American College of Radiology Thyroid Imaging, Reporting and Data System. Thyroid Off J Am Thyroid Assoc (2018) 28:1024–33. doi: 10.1089/thy.2017.0525

CrossRef Full Text | Google Scholar

52. Collins GS, Reitsma JB, Altman DG, Moons KG. Correction: Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD): The TRIPOD Statement. Ann Internal Med (2015) 162:55–63. doi: 10.7326/m14-0697

CrossRef Full Text | Google Scholar

53. Localio AR, Goodman S. Beyond the Usual Prediction Accuracy Metrics: Reporting Results for Clinical Decision Making. Ann Internal Med (2012) 157:294–5. doi: 10.7326/0003-4819-157-4-201208210-00014

CrossRef Full Text | Google Scholar