The use of modified TI-RADS using contrast-enhanced ultrasound features for classification purposes in the differential diagnosis of benign and malignant thyroid nodules: A prospective and multi-center study

Objectives: To evaluate the diagnostic efficacy of a modified thyroid imaging reporting and data system (TI-RADS) in combination with contrast-enhanced ultrasound (CEUS) for differentiating between benign and malignant thyroid nodules and to assess inter-observer concordance between different observers.

Methods: This study included 3353 patients who underwent thyroid ultrasound (US) and CEUS in ten multi-centers between September 2018 and March 2020. Based on a modified TI-RADS classification using the CEUS enhancement pattern of thyroid lesions, ten radiologists analyzed all US and CEUS examinations independently and assigned a TI-RADS category to each thyroid nodule. Pathology was the reference standard for determining the diagnostic performance (accuracy (ACC), sensitivity (SEN), specificity (SPN), positive predictive value (PPV), and negative predictive value (NPV)) of the modified TI-RADS for predicting malignant thyroid nodules. The risk of malignancy was stratified for each TI-RADS category-based on the total number of benign and malignant lesions in that category. ROC curve was used to determine the cut-off value and the area under the curve (AUC). Cohen’s Kappa statistic was applied to assess the inter-observer agreement of each sonological feature and TI-RADS category for thyroid nodules.

Results: The calculated malignancy risk in the modified TI-RADS categories 5, 4b, 4a, 3 and 2 nodules was 95.4%, 86.0%, 12.0%, 4.1% and 0%, respectively. The malignancy risk for the five categories was in agreement with the suggested malignancy risk. The ROC curve showed that the AUC under the ROC curve was 0.936, and the cutoff value of the modified TI-RADS classification was >TI-RADS 4a, whose SEN, ACC, PPV, NPV and SPN were 93.6%, 91.9%, 90.4%, 93.7% and 88.5% respectively. The Kappa value for taller than wide, microcalcification, marked hypoechoic, solid composition, irregular margins and enhancement pattern of CEUS was 0.94, 0.93, 0.75, 0.89, 0.86 and 0.81, respectively. There was also good agreement between the observers with regards to the modified TI-RADS classification, the Kappa value was 0.80.

Conclusions: The actual risk of malignancy according to the modified TI-RADS concurred with the suggested risk of malignancy. Inter-observer agreement for the modified TI-RADS category was good, thus suggesting that this classification was very suitable for clinical application.

Introduction

With the development and wide application of high resolution ultrasound, the detection rate of thyroid nodules has increased significantly (1–3). Although there is a high prevalence of thyroid nodules, only 1.6% to 12% of these are malignant (4, 5). According to the bethesda classification system, only 3%−7% of the thyroid nodules undergoing fine-needle aspiration (FNA) have clearly malignant features, at least 60%–70% of thyroid nodules are proven to be benign via pathological analysis. The pathological type of thyroid nodules directly affects the treatment and prognosis of patients. Therefore, the accurate judgment of benign and malignant thyroid nodules is of important clinical significance.

Ultrasound (US) is a simple and reproducible non-invasive method and remains the modality of choice for patients with thyroid nodules. US can distinguish benign and malignant tumors by specific ultrasound imaging characteristics (6, 7). Usually, the suspicious signs of malignant nodules on US include a solid composition, a taller shape rather than a wider shape, an irregular margin, micro-calcification, and marked hypo-echogenicity (8, 9). However, the grey scale and Doppler US features of benign and malignant nodules overlap. Furthermore, a single ultrasound sign cannot reliably predict benign and malignant thyroid nodules (10). Therefore, prediction models have been developed for malignancy that combine multiple US features to improve the accuracy of diagnosing benign and malignant thyroid nodules. In 2009, Horvath et al. were the first to classify thyroid nodules based on the principles that have been used in the breast imaging reporting and data system of the American College of Radiology using ten malignant-related ultrasound features, and proposed the first thyroid imaging reporting and data system (TI-RADS) classification system (11). As a quantitative system for the risk stratification of malignant tumors in thyroid nodules, however, this sonographic model is not applicable to all thyroid nodules and is difficult to apply. In the same year, Park et al. proposed a multi-factor logistic regression analysis equation to predict the malignant probability of thyroid nodules based on 12 types of sonographic features (12). However, this prediction equation is more complicated and difficult to apply. To overcome these limitations, Kwak et al. used several suspicious sonographic features and calculated the fitting probability of malignant tumors (13). The Park equation and the Horvath TI-RADS includes more suspicious malignant features and sonographic patterns and are relatively complex. The model proposed by Kwak et al. simplified the number of suspicious malignant signs on US to five. Subsequently, the TI-RADS became widely used in clinical practice.

Over recent years, the development of new US technology has improved the diagnostic accuracy of thyroid nodules, especially the application of contrast-enhanced ultrasound (CEUS). CEUS can be applied non-invasively in real-time and continuously evaluate the perfusion of the microvessels in thyroid nodules under high frequency US (14). Studies have shown that the combination of the TI-RADS classification with CEUS significantly improves the diagnostic accuracy of thyroid nodules (15, 16). However, CEUS enhancement features for thyroid nodules has still not been fully implemented with the TI-RADS system. In 2017, we proposed a new classification standard based on the TI-RADS classification criteria proposed by Horvath et al. (11), Park et al. (12) and Kawk et al. (13), that combined the five ultrasound signs proposed by Kawk TI-RADS (13), and combined it with CEUS enhancement to form a modified version of TI-RADS (17).

We showed that the modified version of TI-RADS significantly increased diagnostic accuracy for the identification of thyroid nodules, particularly for TI-RADS 4a and 4b lesions. This modified version of TI-RADS was validated by a single center, but all cases were retrospective studies. Thus, the aim of the current study was to confirm the diagnostic efficacy and assess the inter-observer agreement for thyroid nodule characterization using the modified version of TI-RADS in prospective, multi-center trial.

Materials and methods

This prospective multi-center study was approved by the Institutional Review Boards of the ten participating centers

Study population

Between September 2018 and March 2020, we initially collected 3822 consecutive patients from ten centers. The inclusion criteria were as follows: (1) patients with clinically suspected thyroid nodules, (2) patients who consented to undergo CEUS and (3) patients with a final pathological diagnosis as determined by surgical pathology or cytopathological results based on the Bethesda system. Patients were excluded if they refused to undergo final pathological diagnosis or had non-diagnostic or indeterminate cytological results for a lesion without surgical confirmation. Finally, our study featured 3353 patients with 4532 thyroidal nodules. Of the 3353 participants, 729 were male and 2624 were female, patient age ranged from 18 to 82 years with a mean of 46.1 ± 12.2 years). The flow chart of our study is illustrated in Figure 1.

FIGURE 1

Figure 1 Flow chart of the study population. TNs, thyroid nodules; US, ultrasound; CEUS, contrast-enhanced ultrasound; FNA, fine needle aspiration.

Conventional ultrasound and contrast-enhanced ultrasound examination

All patients underwent conventional ultrasound examinations and CEUS analysis. For conventional US examination, we used a linear, high-frequency probe. Patients were positioned in a supine position with dorsal flexion of the head. Thyroid nodules were then evaluated for location, size, echogenicity, internal composition, margin, shape and the presence/absence of micro-calcification. The internal component of each nodule was classified as solid, mixed or cystic. Echogenicity was classified as hyper-echogenicity, iso-echogenicity, hypo-echogenicity or marked hypo-echogenicity. The margins were classified as irregular or regular. Calcifications, when present, were categorized as micro-calcification (equal to or < 1mm in diameter) or macrocalcification (> 1mm). If a nodule showed both microcalcification and macrocalcification, it was classified as microcalcification. Shape was categorized as taller than wide (greater in its anteroposterior dimension than in its transverse dimension) or wider than tall.

Before starting the multi-center study, all hospitals participating in the center were trained on the specific classification methods and standards to establish a unified approach. Ten experienced radiologists used the TI-RADS classification criteria to classify thyroid nodules according to five ultrasound signs (solid component, marked hypo-echogenicity, taller than wide shape, microcalcification and irregular margin) to evaluate each nodule. This was performed in a blind and independent manner. The TI-RADS classification criteria were as follows (17): TI-RADS score 1: normal thyroid; TI-RADS score 2: no malignant sign, benign lesions; TI-RADS score 3: one malignant sign, high probability of being benign; TI-RADS score 4a: two malignant signs, possibly benign; TI-RADS score 4b: three malignant signs, high probability of malignancy; TI-RADS score 5: four to five malignant signs, highly suggestive of malignancy.

Modified TI-RADS diagnostic criteria in combination with CEUS

The contrast agent used in this study was SonoVue (Bracco, Milan, Italy). A 20-G needle was inserted into the peripheral veins to establish intravenous access. Twenty-five mg of SonoVue was diluted in 5 mL of saline and vibrated for 30s to create a microbubble suspension. The suspension was then injected as a bolus and each 2.4 mL injection was then flushed with 5 mL of saline. The dynamic perfusion of the lesion was continuously observed in real time. The CEUS diagnostic criteria were divided into circular enhancement, high enhancement, equal enhancement and low enhancement when compared to the surrounding thyroidal parenchyma.

If CEUS indicated high enhancement or circular enhancement, then the TI-RADS score was reduced by one, if the initial score was 2, then the score remained the same (Figures 2–4). If CEUS indicated low enhancement, then the TI-RADS score was increased by one, a score of 5 remained the same (Figures 5–7). If CEUS indicated equal enhancement, then the TI-RADS classification remained the same. With regards to the modified TI-RADS classification, scores 2-4a were diagnosed as benign and scores 4b-5 were diagnosed as malignant (Table 1).

FIGURE 2

Figure 2 The case of a 46-year-old woman with a 37.3 × 18.1 × 24.7 mm solid hyper-echoic nodule in the left lobe of the thyroid. (A) Conventional two-dimensional image showed that the nodule had one malignant indicator (solid) and was classified with a TI-RADS score of 3. (B) Ultrasound contrast image showing ring enhancement. The modified version of TI-RADS combined with CEUS returned a score of 2 and the patient was diagnosed with a benign nodule. (C) Pathological image of the lesion, a nodular goiter.

FIGURE 3

Figure 3 The case of a 31-year-old woman with a 15.1 × 10.1 × 9.8 mm solid hypo-echoic nodule in the right lobe of the thyroid. (A) Conventional ultrasound showed that the nodule had one malignant indicator (solid) and was classified as a TI-RADS score of 3. (B) Contrast-enhanced ultrasound detected high enhancement. The modified TI-RADS resulted in a score of 2 and indicated a benign nodule. (C) Pathology of the lesion showed an adenoma.

FIGURE 4

Figure 4 The case of a 64-year-old woman with a 10.6 × 6.7 × 5.9 mm solid mark hypo-echoic nodule in the right lobe of the thyroid. (A) Conventional two-dimensional image showing that the nodule had three malignant indicators (solid, mark hypo-echoic and irregular margin) and was classified with a TI-RADS score of 4b. (B) Ultrasound contrast image showing high enhancement. The modified version of the TI-RADS combined with CEUS returned a score of 4a, and the diagnosis was a benign nodule. (C) Pathological image of the lesion, a nodular goiter.

FIGURE 5

Figure 5 The case of a 43-year-old woman with a 15.2 × 11.1 × 12.5 mm solid hypo-echoic nodule in the right lobe of the thyroid. (A) Conventional two-dimensional image showing that the nodule had three malignant indicators (solid, irregular margin and microcalcifications) and was classified with a TI-RADS score of 4b. (B) Ultrasound contrast image showing low enhancement. The modified version of TI-RADS combined with CEUS returned a score of 5, and the patient was diagnosed with a malignant nodule. (C) Pathological image of the lesion, a PTC.

FIGURE 6

Figure 6 The case of a 47-year-old man with a 7.8 × 6.8 × 7.5 mm solid mark hypo-echoic nodule in the right lobe of the thyroid. (A) Conventional two-dimensional image showing that the nodule had two malignant indicators (solid, mark hypoechoic) and was classified with a TI-RADS score of 4a. (B) Ultrasound contrast image showing low enhancement. The modified version of the TI-RADS combined with CEUS returned a score of 4b and the diagnosis was a malignant nodule. (C) Pathological image of the lesion, a PTC.

FIGURE 7

Figure 7 The case of a 51-year-old woman with a 20.4 × 15.1 × 16.9 mm solid mark hypo-echoic nodule in the right lobe of the thyroid. (A) Conventional two-dimensional image showing that the nodule had three malignant indicators (solid, mark hypo-echoic and irregular margin) and was classified with a TI-RADS score of 4b. (B) Ultrasound contrast image showing low enhancement. The modified version of the TI-RADS combined with CEUS returned a score of 5, and the diagnosis indicated a malignant nodule. (C) Pathological image of the lesion, a PTC.

TABLE 1

Table 1 Modified TI-RADS diagnostic criteria in combination with CEUS.

US-guided FNA procedures

US-guided FNA was performed with 23-gauge needles, each lesion was aspirated three times. Materials obtained from aspiration biopsy were expelled onto glass slides and smeared. All smears were placed immediately in 95% alcohol for Papanicolaou staining. The interpretation of FNA was based on the Bethesda system for reporting thyroid cytopathology.

Statistical analysis

Statistical analyses were performed using SPSS for Windows software (version 23.0, SPSS, Chicago, III, USA). Measurement data are given as mean ± standard deviation and count data are given as percentage and frequency. Analysis of receiver operating characteristic (ROC) curves were used to determine the cut-off value, area under the curve (AUC), and 95% confidence interval (CI). We calculated the ACC, SEN, SPN, PPV and NPV of the modified TI-RADS system to identify malignant thyroid nodules. The level of significance was defined as p < 0.05. Cohen’s Kappa (κ) coefficient was determined separately to evaluate inter−observer agreement for each of the TI-RADS malignant features. The κ values were interpreted as follows: 0.01–0.20 (poor agreement), 0.21–0.40 (fair agreement), 0.41–0.60 (moderate agreement), 0.61–0.80 (good agreement) and 0.81–1.0 (very good agreement).

Results

Nodule diagnosis

The final diagnosis of the 4532 nodules was benign in 2150 (47.4%) nodules and malignant in 2382 (52.6%) nodules. Final diagnoses were determined by surgical resection in 1598 of the 2382 malignant nodules, including 1529 papillary thyroid carcinomas (PTC), 39 follicular carcinomas, 14 cases of focal canceration of nodular goiter, 7 medullary thyroid carcinomas (MTC), 4 anaplastic thyroid carcinomas (ATC), 3 metastatic carcinomas and 2 lymphomas. In total, 784 of the malignant nodules diagnosed by FNA were PTC. The 1376 surgically confirmed benign nodules included 186 adenomas, 990 nodular goiter, 148 Hashimoto’s thyroiditis, and 52 cases of subacute thyroiditis. Overall, 774 benign nodules were diagnosed based on repetitive benign FNA results or benign FNA results by US follow-up studies.

US features of thyroid nodules

The mean maximum diameter of nodules was 13.6 ± 11.8 mm (range: 4.5–59.0 mm) and the mean size of benign nodules was 18.4 ± 14.0 mm, this was significantly larger than that of malignant nodules (9.3 ± 7.1 mm, p < 0.001). Of the 4532 thyroid nodules, there were 3673 solid composition nodules and 859 mixed composition nodules, 413 marked hypoechoic nodules, 2021 hypoechoic nodules, 473 isoechoic nodules and 1625 hyperechoic nodules. We identified 1453 nodules with irregular margins and 3079 nodules with regular margins, 1733 nodules with microcalcifications, 341 nodules with macrocalcifications and 2458 nodules with no calcifications. There were 3339 wider than tall nodules and 1193 taller than wide nodules (Table 2).

TABLE 2

Table 2 US features of thyroid nodules.

Malignancy risk according to category in the modified TI-RADS

Of the 4352 thyroid nodules assessed, 159 (3.5%) were classified as TI-RADS 2, 1256 (27.7%) as TI-RADS 3, 649 (14.3%) as TI-RADS 4a, 1323 (29.2%) as TI-RADS 4b and 1145 (25.3%) as TI-RADS 5. Of the 159 thyroid nodules categorized as TI-RADS 2, none were malignant (0%). Of the 1256 thyroid nodules categorized as TI-RADS 3, 52 were malignant (4.1%). Of the 649 thyroid nodules categorized as TI-RADS 4a, 78 were malignant (12.0%). Of the 1323 thyroid nodules categorized as TI-RADS 4b, 1138 were malignant (86.0%). Of the 1145 thyroid nodules categorized as TI-RADS 5, 1092 were malignant (95.4%). The calculated malignancy risk in the modified TI-RADS categories 5, 4b, 4a, 3 and 2 nodules was 95.4%, 86.0%, 12.0%, 4.1% and 0%, respectively, and all were estimated within the range of the suggested malignancy risk in the modified TI-RADS (Table 3).

TABLE 3

Table 3 Malignancy risk according to category in the modified version of the TI-RADS.

Diagnostic performance of the modified version of the TI-RADS for predicting malignant thyroid nodules

ROC curve analysis was used to analyze the diagnostic efficacy of the modified TI-RADS classification for differentiating benign and malignant thyroid nodules Figure 8. The AUC under the ROC curve was 0.936 (95% CI: 0.928–0.943, p < 0.01) and the best cut-off value for predicting malignant thyroid nodules was > TI-RADS 4a. Considering TI-RADS 4b and TI-RADS 5 together as predictors for malignancy, the SEN, ACC, PPV, NPV and SPN were 93.6%, 91.9%, 90.4%, 93.7% and 88.5%, respectively.

FIGURE 8

Figure 8 ROC analyses for the diagnostic performance of the modified version of the TI-RADS for predicting the malignancy of thyroid nodules. The best cut-off was > TIRADS 4a, resulting in 93.6% SEN and 88.5% SPN.

Inter-observer agreement

We calculated Cohen’s Kappa value for each of the five US features and the CEUS enhancement pattern (Table 4). The highest inter-observer agreement was observed for the taller than wide shape and for microcalcification, the Kappa value for these two features was 0.94 and 0.93, respectively. The Kappa value for marked hypoechoic, solid composition, irregular margins and enhancement pattern of the CEUS was 0.75, 0.89, 0.86 and 0.81, respectively. There was also good agreement between the observers for the modified TI-RADS classification, the Kappa value was 0.80, thus implying that the modified TI-RADS system showed comparable results when used for the analysis of thyroid nodules by different radiologists.

TABLE 4

Table 4 The inter-observer agreement for US features and TI-RADS categorization for the diagnosis of thyroid nodules.

Discussion

TI-RADS is a quantitative scoring method that has been developed over recent years. This system can stratify the risk of malignancy for thyroid nodules and standardize the US reports for the thyroid. Consequently, this method is an effective form of communication between clinicians and pathologists. Since Horvath et al. (11) first proposed TI-RADS as a quantitative system for the risk stratification of thyroid nodules in 2009, its format and content have evolved and undergone significant development. Many researchers (12, 13, 17–19) have proposed different TI-RADS classification systems which have been used for the effective management of US for thyroid nodules. However, despite these efforts, there were many different versions and complex models of the TI-RADS classification and there was no unified classification standard. Furthermore, there were different guidelines for TI-RADS in different regions and countries, and there were certain differences between different guidelines this meant that the system was not widely adopted across the world. Therefore, there has been many attempts to develop a practical and standardized risk stratification system for thyroid nodules so as to provide consistent management strategies for assessing thyroid nodules in clinical practice (20).

When facilitated by micro-bubble contrast agents, CEUS can display microvessels, large vessels and dynamic perfusion simultaneously. Compared with conventional ultrasound, CEUS can reveal better characteristics of focal thyroid nodules (21). At present, CEUS is widely used for the differential diagnosis of benign and malignant thyroid nodules. Zhao et al. (15) showed that CEUS has high value for the differentiation of benign and malignant thyroid nodules, and was significantly more useful than conventional ultrasound. The results of the present study showed that benign thyroid nodules mainly showed ring enhancement, high enhancement or equal enhancement, while malignant nodules mainly showed low enhancement, especially non-homogeneous low enhancement, these findings were similar to those of previous studies (22). In this study, low enhancement was used as the standard to judge malignant nodules. The SEN, SPN, ACC, PPV and NPV of benign and malignant thyroid nodules diagnosed by CEUS were 82.8%, 81.6%, 82.3%, 83.0% and 81.4%, respectively. Low enhancement was considered to be the main enhancement mode for thyroid malignant nodule CEUS (23–25). Compared with the low enhancement mode of malignant nodules, benign nodules mainly showed high enhancement, equal enhancement and ring enhancement (26–28).

Many studies have shown that the combination of CEUS and TI-RADS classification for conventional ultrasound can improve the accuracy of diagnosing benign and malignant thyroid nodules. Zhao et al. (15) retrospectively analyzed the conventional ultrasound and CEUS enhancement characteristics of 117 cases of thyroid nodules and compared the diagnostic efficiency of TI-RADS alone against CEUS combined with TI-RADS for predicting benign and malignant thyroid nodules. The results showed that the ACC, SEN, SPN, PPV and NPV for TI-RADS + CEUS were the highest and were significantly higher than that for TI-RADS or CEUS alone. Ruan et al. (29) constructed a CEUS TI-RADS by adding CEUS to widely accepted nonenhanced US features, the CEUS TI-RADS showed the highest AUC under the ROC curve comparison with all other systems (AUC=0.93, P<0.001), the highest biopsy yield of malignancy at 66% (157 of 239 nodules), and the lowest unnecessary biopsy rate at 34% (82 of 239 nodules). Our previous retrospective study (17) of 298 thyroid nodules in 206 patients showed that the SEN, SPN, ACC, PPV and NPV acquired by CEUS combined with TI-RADS were 96.3%, 94.7%, 95.0%, 80.0% and 99.1%, respectively. Furthermore, the diagnostic efficiency of CEUS for judging benign and malignant thyroid nodules was significantly higher than that of TI-RADS or CEUS alone.

Many studies have stratified the risk of each TI-RADS category separately. Although there were some differences between these studies, they all reported a common pattern, with the risk of malignancy increasing from the TI-RADS 2 to the TI-RADS 5 category. The risk of malignancy described by Horvath et al. (11) was 0, < 5%, 5%–10%, 10%–80% and > 80%, respectively for TI-RADS 2, 3, 4a, 4b, and 5 lesions. Horvath et al. prospectively verified the diagnostic value of this TI-RADS classification for evaluating 1097 benign and malignant thyroid nodules. This previous study showed that the SEN, SPN, PPV, NPV and ACC were 88%, 49%, 49%, 88% and 94%, respectively. The risks of malignancy reported by Park et al. (12) was 0%–7%, 8%–23%, 24%–50%, 51%–90% and 91%–100%, respectively for TI-RADS 1, 2, 3, 4 and 5 lesions. Wang et al. (30) reported that the SEN, SPN, ACC, PPV and NPV when using Park’s TI-RADS system were 96.8%, 71.3%, 83.0%, 74.1% and 96.3%, respectively. The risks of malignancy reported by Kwak et al. (13) were 0%, 2–2.8%, 3.6–12.7%, 6.8–37.8%, 21%–91.9% and 88.7%–97.9%, respectively, for TI-RADS2, 3, 4a, 4b, 4c and 5 lesions. Zhang et al. (17) reported that the SEN, SPN, PPV, NPV and ACC were 94.4%, 69.3%, 40.5%, 98.3% and 73.8%, respectively. In our present study, the calculated risk of malignancy for the modified TI-RADS categories 5, 4b, 4a, 3 and 2 nodules were 95.4%, 86.0%, 12.0%, 4.1% and 0%, respectively. The risk of malignancy risk for the five categories was within the range of the suggested risk of malignancy. Therefore, the modified TI-RADS can be applied for the qualitative diagnosis of benign and malignant thyroid nodules.

Based on ROC curve analyses, our study revealed an AUC of 0.936, the best cut-off value for predicting malignant thyroid nodules was > TI-RADS4a. When considering TI-RADS 4b and TI-RADS 5 together as predictors for malignancy, the SEN, ACC, PPV, NPV and SPN were 93.6%, 91.9%, 90.4%, 93.7% and 88.5%, respectively. In our study, the modified TI-RADS had high diagnostic efficiency for the diagnosis of benign and malignant thyroid nodules. Of the 4352 thyroid nodules assessed, 159 were classified as TI-RADS 2, none were malignant and the diagnostic accordance rate was 100%. In total, 1256 were classified as TI-RADS 3, of which 1204 were benign nodules; the diagnostic accordance rate was 95.9%. In total, 649 were classified as TI-RADS 4a, of which 571 were benign nodules; the diagnostic accordance rate was 88.0%. In total, 1323 were classified as TI-RADS 4b, of which 1138 were malignant nodules; the diagnostic accordance rate was 86.0%. In total, 1145 were classified as TI-RADS 5, of which 1092 were malignant nodules; the diagnostic accordance rate was 95.4%. Except for TI-RADS 2 nodules, there was a certain misdiagnosis rate in other classified nodules. There are several potential reasons for these findings. First, as with conventional two-dimensional ultrasound, CEUS enhancement patterns for some benign and malignant nodules can also overlap. For example, some malignant nodules had a rich blood supply and showed high enhancement, while some benign nodules may exhibit scar hyperplasia or fibrous tissue hyperplasia, thus resulting in low enhancement. Second, some nodules were so small that it was difficult to judge their CEUS enhancement mode. Third, some benign nodules were often associated with focal PTC or FTC carcinogenesis; therefore, it was difficult to correctly diagnose these focal forms of carcinogenesis. Fourth, the presence of hidden PTMC in thyroid glands with diffuse lesions was not typical on conventional two-dimensional ultrasound or CEUS, thus increasing the difficulty of diagnosis. Finally, some thyroid inflammatory lesions were similar to malignant lesions in conventional two-dimensional ultrasound or CEUS enhancement mode; it was difficult to correctly diagnose these inflammatory lesions.

The modified TI-RADS is simple and accurate for the evaluation of benign and malignant thyroid nodules. The radiologists only need to accurately evaluate the conventional two dimensional ultrasound signs and CEUS enhancement mode to classify nodules. Theoretical malignant risk and clinical treatment suggestions were also given for each classification of nodules, thus allowing better communication between clinicians and pathologists. Another important role of the modified TI-RADS was to standardize the criteria for different radiologists to evaluate the signs of thyroid nodules. Therefore, the classification system should have good consistency and repeatability among different radiologists (31–33). Our study revealed strong inter-observer agreement between different radiologists when using the modified TI-RADS categories and features for thyroid nodule characterization. We found that the highest inter-observer agreement was for shape and micro-calcification. The Kappa value for these two features was 0.94 and 0.93, respectively, thus showing excellent consistency among the different observers. There was also good consistency among the observers for echogenicity, composition, margins and enhancement mode; the Kappa values were 0.75, 0.89, 0.86 and 0.81, respectively. In addition, different observers showed good consistency when using the modified TI-RADS classification system; the Kappa value was 0.80. These results showed that the modified TI-RADS produced comparable results for the analysis of suspicious thyroid nodules when used by different radiologists in thyroid imaging.

This study had some limitations that need to be considered. First, all US examinations were performed and analyzed by highly experienced radiologists. Further studies relating to the performance of this reporting system when applied by less experienced radiologists may be needed. Second, we did not compare the modified TI-RADS to the other TI-RADS classification systems. Third, we only evaluated inter-observer consistency, we did not evaluate intra-observer consistency, this needs to be addressed by future studies.

Conclusion

The modified TI-RADS had high diagnostic efficiency for the diagnosis of benign and malignant thyroid nodules. The inter-observer agreement for the modified TI-RADS category was excellent, thus suggesting that this classification is very suitable for clinical application.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.

Author contributions

PiZ and FC have equal contributions to this article. Drafting the work or revising: WL, PiZ. Acquisition, analysis, or interpretation of data: WL, FC, PeZ, LX, LW, ZW, YY, XL, BW, WY, HZ, YT. Conception or design: WL, PiZ. Final approval of the manuscript: PiZ.

Funding

This research was supported by grants from the National Natural Science Foundation of China (Reference: 81871367), the Natural Science Foundation of Hunan Province, China (Reference: 2022JJ30894) and the Project of Hunan Provincial Health Commission (Reference: 202209025123).

Acknowledgments

We thank Prof. Martin Krix, Hanshu Li and Caixiang Xia saw for their constructive suggestions to improve our manuscript. We thank Lifang Li and Yan Liu for assistance in intravenous injection of the contrast agent. We thank Yang Liu, Yu Xin and Sheng Xiao for assistance in cytology. We thank Xinghao Zhang, Xiangqi Zhou and Zhen Huang for assistance in statistical analysis.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Londero SC, Krogdahl A, Bastholt L, Overgaard J, Trolle W, Henrik Baymler Pedersen HB, et al. Papillary thyroid microcarcinoma in Denmark 1996-2008: a national study of epidemiology and clinical significance. Thyroid (2013) 23(9):1159–64. doi: 10.1089/thy.2012.0595

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Jemal A, Siegel R, Xu J, Ward E. Cancer statistics, 2010. CA Cancer J Clin (2010) 60:277–300. doi: 10.3322/caac.20073

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Magreni A, Bann DV, Schubart JR, Goldenberg D. The effects of race and ethnicity on thyroid cancer incidence. JAMA Otolaryngol Head Neck Surg (2015) 141:319–23. doi: 10.1001/jamaoto.2014.3740

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Smith-Bindman R, Lebda P, Feldstein VA, Sellami D, Goldstein RB, Brasic N, et al. Risk of thyroid cancer based on thyroid ultrasound imaging characteristics: results of a population-based study. JAMA Intern Med (2013) 173:1788–96. doi: 10.1001/jamainternmed.2013.9245

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Uppal A, White MG, Nagar S, Aschebrook-Kilfoy B, Chang PJ, Angelos P, et al. Benign and malignant thyroid incidentalomas are rare in routine clinical practice: a review of 97908 imaging studies. Cancer Epidemiol Biomarkers Prev (2015) 24:1327–31. doi: 10.1158/1055-9965.EPI-15-0292

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Iannuccilli JD, Cronan JJ, Monchik JM. Risk f or malignancy of thyroid nodules as assessed by sonographic crit eria: the need for biopsy. J Ultrasound Med (2004) 23(11):1455–64. doi: 10.7863/jum.2004.23.11.1455

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Cooper DS, Doherty GM, Haugen BR, Kloos RT, Lee SL, Mandel SJ, et al. Revised American thyroid association management guidelines for patients with thyroid nodules and differentiated thyroid cancer. Thyroid (2009) 19(11):1167–214. doi: 10.1089/thy.2009.0110

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Cheng SP, Lee JJ, Lin JL, Chuang SM, Chien MN, Liu CL. Characterization of thyroid nodules using the proposed thyroid imaging reporting and data system (TI-RADS). Head Neck (2013) 35:541–7. doi: 10.1002/hed.22985

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Kim GR, Kim MH, Moon HJ, Chung WY, Kwak JY, Kim EK. Sonographic characteristics suggesting papillary thyroid carcinoma according to nodule size. Ann Surg Oncol (2013) 20:906–13. doi: 10.1245/s10434-012-2830-4

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Remonti LR, Kramer CK, Leitao CB, Pinto LCF, Gross JL. Thyroid ultrasound features and risk of carcinoma: a systematic review and meta-analysis of observational studies. Thyroid (2015) 25(5):538–50. doi: 10.1089/thy.2014.0353

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Horvath E, Majlis S, Rossi R, Franco C, Niedmann JP, Castro A, et al. An ultrasonogram reporting system for thyroid nodules stratifying cancer risk for clinical management. J Clin Endocrinol Metab (2009) 94(5):1748–51. doi: 10.1210/jc.2008-1724

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Park JY, Lee HJ, Jang HW, Kim HK, Yi JH, Lee W, et al. A proposal for a thyroid imaging reporting and data system for ultrasound features of thyroid carcinoma. Thyroid (2009) 19(11):1258–64. doi: 10.1089/thy.2008.0021

CrossRef Full Text | Google Scholar

13. Kwak JY, Han KH, Yoon JH, Moon HJ, Son EJ, Park SH, et al. Thyroid imaging reporting and data system for US features of nodules: a step in establishing better stratification of cancer risk. Radiology (2011) 260(3):892–9. doi: 10.1148/radiol.11110206

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Cosgrove D. Future prospects for SonoVue and CPS. Eur Radiol (2004) 14:116–24. doi: 10.1007/s10406-004-0084-3

CrossRef Full Text | Google Scholar

15. Zhao H, Liu X, Lei B, Chen P, Li J, Wu Y, et al. Diagnostic performance of thyroid imaging reporting and data system (TI-RADS) alone and in combination with contrast-enhanced ultrasonography for the characterization of thyroid nodules. Clin Hemorheol Microcirc (2019) 72(1):95–106. doi: 10.3233/CH-180457

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Huang Y, Hong Y, Xu W, Song K, Huang P. Contrast-enhanced ultrasound improves the accuracy of the ACR TI-RADS in the diagnosis of thyroid nodules located in the isthmus. Ultraschall Med (2022) 43(06):599–607. doi: 10.1055/a-1543-6033

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Zhang Y, Zhou P, Tian SM, Zhao YF, Li JL, Li L. Usefulness of combined use of contrast-enhanced ultrasound and TI-RADS classification for the differentiation of benign from malignant lesions of thyroid nodules. Eur Radiol (2017) 27:1527–36. doi: 10.1007/s00330-016-4508-y

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Haugen BR, Alexander EK, Bible KC, Doherty GM, Mandel SJ, Nikiforov YE, et al. American thyroid association management guidelines for adult patients with thyroid nodulesand differentiated thyroid cancer: The American thyroid association guidelines task force on thyroid nodules and differentiated thyroid cancer. Thyroid: Off J Am Thyroid Association,2016 (2015) 26:1–133. doi: 10.1089/thy.2015.0020

CrossRef Full Text | Google Scholar

19. Tessler FN, Middleton WD, Grant EG, Hoang JK, Berland LL, Teefey SA, et al. ACR thyroid imaging, reporting and data system (TI-RADS): White paper of the ACR TI-RADS committee. J Am Coll Radiol (2017) 14:587–95. doi: 10.1016/j.jacr.2017.01.046

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Middleton WD, Teefey SA, Reading CC, Langer J, Beland MD, Szabunio MM, et al. Multi-institutional analysis of thyroid nodule risk stratification using the American college of radiology thyroid imaging reporting and data system. AJR Am J Roentgenol (2017) 208(6):1331–41. doi: 10.2214/AJR.16.17613

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Sidhu PS, Cantisani V, Dietrich CF, Gilja OH, Saftoiu A, Bartels E, et al. The EFSUMB guidelines and recommendations for the clinical practice of contrast-enhanced ultrasound (CEUS) in nonhepatic applications: update 2017 (long version). Ultraschall Med (2018) 39(2):e2–e44. doi: 10.1055/a-0586-1107

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Deng J, Zhou P, Tian SM, Zhang L, Li JL, Qian Y. Comparison of diagnostic efficacy of contrast-enhanced ultrasound, acoustic radiation force impulse imaging, and their combined use in differentiating focal solid thyroid nodules. PloS One (2014) 9(3):e90674. doi: 10.1371/journal.pone.0090674

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Schleder S, Janke M, Agha A, Schacherer D, Hornung M, Schlitt HJ, et al. Preoperative differentiation of thyroid adenomas and thyroid carcinomas using high resolution contrast-enhanced ultrasound(CEUS). Clin Hemorheol Microcirc (2015) 61(1):13–22. doi: 10.3233/CH-141848

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Zhang B, Jiang YX, Liu JB, Yang M, Dai Q, Zhu QL, et al. Utility of contrast-enhanced ultrasound for evaluation of thyroid nodules. Thyroid Off J Am Thyroid Assoc (2010) 20(1):51–7. doi: 10.1089/thy.2009.0045

CrossRef Full Text | Google Scholar

25. Zhao RN, Zhang B, Yang X, Jiang YX, Lai XJ, Zhang XY. Logistic regression analysis of contrast-enhanced ultrasound and conventional ultrasound characteristics of sub-centimeter thyroid nodules. Ultrasound Med Biol (2015) 41:3102–08. doi: 10.1016/j.ultrasmedbio.2015.04.026

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Li FS, Luo HT. Comparative study of thyroid puncture biopsy guided by contrast- enhanced ultrasonography and conventional ultrasound. Exp Ther Med (2013) 5(5):1381–84. doi: 10.3892/etm.2013.1016

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Bartolotta TV, Midiri M, Galia M, Runza G, Attard M, Savoia G, et al. Qualitative and quantitative evaluation of solitary thyroid nodules with contrast-enhanced ultrasound: initial results. Eur Radiol (2006) 16(10):2234–41. doi: 10.1007/s00330-006-0229-y

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Cantisani V, Consorti F, Guerrisi A, Guerrisi I, Ricci P, Segni MD, et al. Prospective comparative evaluation of quantitative-elastosonography (Q-elastography) and contrast-enhanced ultrasound for the evaluation of thyroid nodules: Preliminary experience. Eur J Radiol (2013) 82(11):1892–8. doi: 10.1016/j.ejrad.2013.07.005

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Ruan J, Xu X, Cai Y, Zeng H, Luo M, Zhang W, et al. A practical CEUS phyroid reporting system for thyroid nodules. Radiology (2022) 305(1):149–59. doi: 10.1148/radiol.212319

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Wang Y, Lei KR, He YP, Li XL, Ren WW, Zhao CK, et al. Malignancy risk stratification of thyroid nodules: Comparisons of four ultrasound thyroid imaging reporting and data systems in surgically resected nodules. Sci Rep (2017) 7(1):11560. doi: 10.1038/s41598-017-11863-0

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Chung R, Rosenkrantz AB, Bennett GL, Dane B, Jacobs JE, Slywotzky C, et al. Interreader concordance of the TI-RADS: Impact of radiologist experience. Am J Roentgenol (2020) 214(5):1152–7. doi: 10.2214/AJR.19.21913

CrossRef Full Text | Google Scholar

32. Teng D, Fu P, Li W, Guo F, Wang H. Learnability and reproducibility of ACR thyroid imaging, reporting and data system (TI-RADS) in postgraduate freshmen. Endocrine (2020) 67(5):643–50. doi: 10.1007/s12020-019-02161-y

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Basha MAA, Alnaggar AA, Refaat R, EI-Maghraby AM, Refaat MM, EIhamed MEA, et al. The validity and reproducibility of the thyroid imaging reporting and data system (TI-RADS) in categorization of thyroid nodules: Multicentre prospective study. Eur J Radiol (2019) 117:184–92. doi: 10.1016/j.ejrad.2019.06.015

PubMed Abstract | CrossRef Full Text | Google Scholar