Risk Stratification in Patients With Follicular Neoplasm on Cytology: Use of Quantitative Characteristics and Sonographic Patterns

Objectives: Differentiating thyroid nodules with a cytological diagnosis of follicular neoplasm remains an issue. The goal of this study was to determine whether ultrasonographic (US) findings obtained preoperatively from the computer-aided detection (CAD) system are sufficient to further stratify the risk of malignancy for this diagnostic cytological category.

Methods: From September 2016 to September 2018 in our hospital, patients diagnosed with Bethesda category IV (follicular neoplasm or suspicion of follicular neoplasm) thyroid nodules and underwent surgical excisions were include in the study. Quantification and analysis of tumor features were performed using CAD software. The US findings of the region of interest, including index of composition, margin, echogenicity, texture, echogenic dots indicative of calcifications, tall and wide orientation, and margin were calculated into computerized values. The nodules were further classified into American Thyroid Association (ATA) and American College of Radiology Thyroid Imaging Reporting & Data System (TI-RADS) categories.

Results: 92 (10.1%) of 913 patients were diagnosed with Bethesda category IV thyroid nodules. In 65 patients, the histological type of the nodule was identified. The quantitative features between patients with benign and malignant conditions differed significantly. The presence of heterogeneous echotexture, blurred margins, or irregular margins was shown to have the highest diagnostic value. The risks of malignancy for nodules classified as having very low to intermediate suspicion ATA, non-ATA, and high suspicion ATA patterns were 9%, 35.7%, and 51.7%, respectively. Meanwhile, the risks of malignancy were 12.5%, 26.1%, and 53.8% for nodules classified as TIRADS 3, 4, and 5, respectively. When compared to human observers, among whom poor agreement was noticeable, the CAD software has shown a higher average accuracy.

Conclusions: For patients with nodules diagnosed as Bethesda category IV, the software-based characterizations of US features, along with the associated ATA patterns and TIRADS system, were shown helpful in the risk stratification of malignancy.

Introduction

Thyroid nodular disease is one of the most common endocrine disorders. The preoperative diagnosis of thyroid nodule is commonly based on fine-needle aspiration cytology (FNAC) results (1). However, follicular cell-derived thyroid nodules have overlapping cytomorphological features (2–4); thus, 15%-30% of aspirations are indeterminate (5). The Bethesda system of thyroid cytopathology defines follicular neoplasm (FN) or suspicion of follicular neoplasm (SFN) as Bethesda category IV, with the goal of increasing the probability of detecting follicular thyroid cancers (6). It reports a 10% to 40% association with malignancy risk, and diagnostic thyroidectomy is often recommended for making a diagnosis (7–9). Moreover, up to 85% of thyroidectomy procedures are performed for benign lesions in this category (10). The ATA guidelines suggest consider using assessment methods for molecular markers, such as mutational testing using next-generation sequencing or special immunohistochemical staining, rather than the operative approach for diagnosis. However, these methods are expensive and currently are not widely available.

High-resolution ultrasonography (US) is the initial imaging technique used to evaluate the gross morphologic characteristics of thyroid nodules. Most studies of the US features of thyroid nodules have focused on how to select patients for further cytological examinations, while few studies have reported the predictive value of US for malignant nodules in Bethesda category IV. In addition, the results of these studies were not conclusive (1, 11, 12). Our previous study showed that certain gray-scale US features may help differentiate pathologically confirmed follicular thyroid cancer (FTC) from follicular adenoma (FA) (13). However, tumors with Bethesda category IV cytology do not always result in a histologic diagnosis of FTC and FA, and it remains uncertain whether they can be used to differentiate this category preoperatively. In addition, algorithms that can identify patients at high risk of malignancy may allow surgeons to perform a single definitive operation (usually lobectomy or total thyroidectomy if needed), thereby saving time, reducing patient morbidity, and reducing costs.

The major limitations of thyroid US are inter-observer and intra-observer variability (14–19). Computerized quantification methods that can characterize the sonographic features to make the diagnosis more objective are available, and have shown acceptable agreement with experienced clinicians (20–26). A computer-aided detection (CAD) system (K180006) had been developed based on these methods and validated following the guidelines of the United States Food and Drug Administration (FDA) for clinical use.

Because further differentiation of Bethesda category IV lesions is challenging, we aimed to examine whether the information provided by commercial CAD to surgeons is sufficient to predict thyroid cancer. Moreover, we examined the usefulness of the current ATA and TIRADS systems in surgical planning for these patients.

Materials and Methods

Participants

The institutional review board of the National Taiwan University Hospital approved the data collection and analyses in this study. Based on thyroid FNACs performed during the study period from September 2016 to September 2018 in the National Taiwan University Hospital, 92 (10.1%) of 913 patients were diagnosed with Bethesda category IV thyroid nodules. Because molecular testing is not widely available in Taiwan, physicians recommended surgical excision to these patients for the removal and definitive diagnosis of an FN/SFN thyroid nodule. Sixty-five patients who underwent thyroidectomy were recruited in this study. The diagnoses were based on the histopathological findings of surgical specimens that were assessed by pathologists. The other 27 patients refused to undergo an operation.

Equipment and Ultrasonographic Procedures

All sonograms were performed using commercial ultrasonography devices with multi-frequency (4-12 megahertz) linear probes set to the highest frequency available, including HDI 5000 (Philips Medical Systems, Bothell, WA, the USA), Voluson 730 (GE Medical Systems, Milwaukee, WI, the USA), and ALOKA ProSound 2 (Hitachi Medical Systems Europe, Steinhausen, Switzerland). The procedure was performed while the participant was in a supine position with the neck hyperextended. Images were captured using the maximum nodule diameter. Image analysis was conducted offline using the DICOM format of images on a separate computer. Quantification of tumor features was performed using a commercial software (AmCAD-UT, AmCad BioMed, Taiwan).

Briefly, the operators provided the four endpoints of the axes on the thyroid nodule margins. The CAD software was used to calculate the contour of the mass, to distinguish it from normal thyroid tissues. In the image analysis session, the US findings of the region of interest (ROI), including index of composition (CoI), index of margin (MI), index of echogenicity (EI), index of texture (TI), index of echogenic dots indicative of calcifications (CaI), index of tall and wide orientation (TWI), and index of margin irregularity (MII), were calculated into computerized values.

CoI calculated the percentage of the cystic component consisting of the anechoic pixels. The anechoic pixels were those pixels with the gray-scale intensity readings below a certain pre-defined threshold inside the ROI (20). CaI was the proportion of the echogenic foci in the solid component of ROI. The solid component was defined to be the area of the ROI excluding the cystic area. Echogenic foci were then represented by those hyperechoic pixels with the gray-scale intensity readings higher than a pre-defined threshold (20). EI was the difference between the average gray-scale intensities of the ROI area and the surrounding reference area outside the ROI (21). TI was to compute the texture heterogeneity of the solid component by calculating the texture variation among small areas within the ROI (22). The higher TI value indicated a greater composition difference of the nodule tissue. TWI calculated the height to width ratio with the nodule height measured along the sound-beam direction and the nodule width in the direction perpendicular to the height as defined by ACR. MI was an indicator of the ill-defined margin that calculated the proportion of the indistinctive border between the nodule and the thyroid parenchyma (27). MII was computed to describe how irregular the nodule margin was by comparing the radius variation of the ROI margin to that of a smooth margin defined by the nodule long and short axes and a corresponding Bezier curve.

These index values were then quantified to represent features that described the thyroid nodule, including factors such as solid or cystic; well-defined or ill-defined; hyperechoic/isoechoic or hypoechoic/markedly hypoechoic; heterogeneous or homogeneous; presence of calcification; presence of taller-than-wide orientation; and presence of irregular margin (20–23).

CAD software was used to classify nodules according to the 2015 ATA sonographic patterns (CAD-ATA), using the referral description of the sonographic features (28). Scenarios not described in the 2015 ATA classification were classified into a separate category referred to as non-ATA patterns. These scenarios included heterogenous nodules with or without other suspicious features, and iso- or hyperechoic nodules with at least one suspicious feature.

CAD software was also utilized to calculate the total risk score of each thyroid nodule by summing the scores of each US feature and classifying them into TIRADS categories (CAD-TIRAD) (29).

Recorded images of thyroid ultrasound were also independently reviewed by three human observers (MH Wu, KY Chen, and MF Wang), all with more than 10 years’ experience in thyroid ultrasound. They interpreted each nodule with 2015 ATA sonographic patterns (Observer-ATA) and TIRADS categories (Observer-TIRAD). If there are discordance in their results, the final interpretation was determined by consensus of two or more observers. When there was no consensus through the independent assessment, the image was reviewed jointly to reach a consensus.

The malignancy rate of each of the ATA and TIRADS category was calculated based on the final histological outcomes of the nodules.

Pathological Evaluation

The decision to biopsy thyroid nodules in our institute was based on 2015 ATA recommendations according to sonographic patterns and sizes. Cytological diagnoses were made using the Bethesda System for Reporting Thyroid Cytopathology by board-certified cytologists, and were retrieved from the cytology reports (6, 30). In our thyroid FNAC practice, cases with inconsistent diagnoses were reviewed and agreement was achieved by discussion.

Statistical Analysis

Data were summarized using descriptive statistics. Categorical variables were presented as numbers and percentages, while continuous variables were presented as means and standard deviations or mediums and ranges for those tested significantly not conforming to normal distributions. Data on patient characteristics were analyzed with Student’s t-test or analysis of variance for continuous variables, Mann-Whitney test for non-normal continuous variables and chi-square test or Fisher’s exact test for categorical variables. Diagnostic performance was estimated by Positive Predictive Value (PPV), Negative Predictive Value (NPV), Sensitivity, and Specificity, all calculated with 95% Confidence Intervals (95% CI). Statistical analyses were performed using SAS 9.4 (Cary, NC, USA). All p-values were two-sided, and the significance level was set at 5%.

Results

Based on thyroid FNACs performed during the study period, 92 (10.1%) of 913 patients were diagnosed with Bethesda category IV thyroid nodules. Among them, 27 patients refused to undergo an operation. There were no significant age, sex, and nodule size differences in those with and without surgeries in this study. In total, 65 patients who underwent surgery were included for further analysis. Of these, 22 had cancer in the thyroid nodule, based on histopathological examination, i.e., a malignancy rate of 33.8% (22/65).

he demographic characteristics of the 65 patients with 65 thyroid nodules are summarized in Table 1. Histopathology results revealed that 19 nodules were nodular goiter; 24 were FA; 15 were papillary thyroid cancer (PTC), including four follicular variant subtypes; and 7 were FTC. None of the patients in this study had Noninvasive follicular thyroid neoplasm with papillary-like nuclear features (NIFTP). The average width, height, and length of the nodules were 2.2 ± 1.3, 1.6 ± 0.8, and 3.3 ± 1.5 cm, respectively.

TABLE 1

Table 1 Demographic characteristics of patients diagnosed with follicular neoplasm on cytology.

The quantitative values of the features between benign and malignant nodules are shown in Figure 1. Significant differences were observed in the feature indices of CaI, EI, TI, TWI, and MII.

FIGURE 1

Figure 1 The quantitative values of the features between benign and malignant nodules.

The demographic and sonographic characteristics of patients with benign and malignant nodules are presented in Table 2. No significant differences were observed in terms of age and sex distribution (p = 0.64 and 0.076, respectively). A higher number of malignant tumors had heterogeneous echotexture (15/22, 68.2% versus 14/43, 32.6%, p = 0.0067) and taller-than-wide orientation (6/22, 27.3% versus 3/43, 7.0%, p = 0.0261).

TABLE 2

Table 2 Comparison of the demographic characteristics of the participants and sonographic characteristics between thyroid nodules with benign and malignant histology.

To learn the preoperative diagnostic value of the US features determined by CAD in differentiating benign from malignant nodules, we calculated the positive and negative predictive values and the sensitivity, specificity, and average accuracy (average sensitivity and specificity), as shown in Table 3. Using a combination of three features, namely, heterogeneous echotexture, irregular margin, and blurred margin, was found to have the highest average accuracy (0.735). 47% of benign Bethesda category IV nodules were with none of these features and 100% of malignant nodules were accurately diagnosed by having at least one of the features present.

TABLE 3

Table 3 Sonographic features used in differentiating benign from malignant conditions in 65 patients with follicular neoplasms.

The nodules were classified into the 2015 ATA sonographic patterns and TIRADS categories based on the sonographic characteristics by both CAD and human observers. For CAD, none of the nodules had an ATA benign pattern. Twenty-two nodules were classified as very low to intermediate suspicion patterns, 29 as high suspicion patterns, and 14 as non-ATA patterns. The risk of finding malignant nodules with very low to intermediate suspicion patterns, non-ATA patterns, and high suspicion patterns were 9%, 35.7%, and 51.7%, respectively. None of the nodules were classified as TIRADS 1 and 2 in our study. Meanwhile, 16, 23, and 26 nodules were classified as TIRADS 3, 4, and 5, respectively. The risk of developing malignant nodules in each TIRADS category increased with increasing TIRADS score: TIRADS 3, 12.5%; TIRADS 4, 26.1%; and TIRADS 5, 53.8% (Table 4).

TABLE 4

Table 4 Malignancy rate based on the ATA and TIRADS category in 65 patients diagnosed with follicular neoplasm on cytology.

For human observers, the intraclass correlation coefficients (ICC) for absolute agreements of the distributions of the 2015 ATA sonographic patterns and TIRADS categories among observers were poor to fair with r = 0.24 [CI 0.09–0.40] and r=0.44[CI 0.29-0.58], respectively (Supplementary Table 1). The risk of finding malignant nodules by observer-consensus with benign to intermediate suspicion patterns, non-ATA patterns, and high suspicion patterns were 16.7%, 0%, and 43.2%, respectively. The malignant rate by observer-consensus was 25% for TIRADS 1-3, 21.2% for TIRADS 4, and 50% TIRADS 5 (Table 4).

The diagnostic values of the ATA/TIRADS determined by CAD and observers are shown in Table 5. The CAD software appeared to have higher average accuracies than the observers (0.675/0.68 and 0.64/0.655, respectively).

TABLE 5

Table 5 Comparison between CAD and Observers.

Discussion

Follicular thyroid neoplasms have numerous clinical and cytological characteristics, and it is difficult to achieve a precise diagnosis preoperatively. The use of clinical manifestations, imaging study results, and even findings of fine-needle aspiration/biopsy of thyroid nodules is not optimal. Resected surgical specimens are important in obtaining an accurate diagnosis of thyroid follicular neoplasms. The ATA guidelines recommend removal and definitive diagnosis of an FN/SFN thyroid nodule, if molecular testing is not performed or inconclusive. However, only 10%-40% of nodules in this category are found to be malignant based on histopathology examination (28, 31). The management of these thyroid nodules, prevention of unnecessary thyroidectomy, and determination of the extent of resection are extremely important in this clinical setting (32). The fact that most patients in the current study have benign diseases based on postoperative histology examination, which is consistent with previous studies (28, 31), justifies the effort to improve the selection of surgery candidates.

Studies have shown that the clinical predictive factors for malignancy in Bethesda category IV nodules include sex and age at diagnosis (5, 33, 34). In this study, we found no difference in terms of sex and age between patients with benign and malignant nodules. Some studies have shown that a larger nodule size is indicative of malignancy. However, the current study and other studies have shown that nodule size is not a predictive factor of malignancy in patients with Bethesda category IV nodules (1, 35–37). Moreover, nodules that were found to be Bethesda category IV by FNAC and were subsequently surgically resected had been previously reported to be larger than those that were not resected (38). However, no significant size differences between nodules with and without surgeries were observed in this study (data not shown). Regarding the pathology distribution, 22 of 62 (33.8%) Bethesda IV nodules were cancers with 15 papillary thyroid cancers being the most common type of cancers and 7 follicular and Hurthle cell carcinomas accounting for about 10% of cases. The distribution appears to be consistent with previous studies (4, 39, 40), where 13~22% of Bethesda IV nodules were PTC or follicular variant PTC and about 9% were follicular or Hurthle cell carcinoma. Although a significant proportion of patients with malignant nodules presented with papillary thyroid cancers, four of them had follicular variants of papillary thyroid cancers.

US is widely used for the screening of thyroid nodules. The presence of US features indicates a malignancy potential for thyroid nodules (41–44). Based on these findings, the nodule can be categorized, and decisions can be made as to whether FNAC or follow-up is required (5, 11, 29, 38). Studies about post-FNAC stratification are limited, and few studies have explored the US features of follicular neoplasm (45, 46). In the current study, the presence of heterogenous echotexture and taller‐than‐wide features was higher in malignant than benign follicular neoplasms. In the past, heterogeneity echotexture was found to be not as important as other sonographic features, and was not listed in the current ATA and TIRDAS category system. However, in the current study and in our previous work, heterogeneous echotexture is associated with an increased risk of malignancy, particularly for follicular neoplasms (13, 22). Studies have shown that a taller-than-wide feature is extremely specific for malignant thyroid nodules. However, information about such features in follicular neoplasms is limited (28, 43, 47, 48).

Because US is a relatively subjective diagnostic method, observers may have different opinions when describing and interpreting lesions (14, 49), leading to poor reliability for some features (14, 16, 17, 19, 50–56). Several studies have shown that inconsistencies in the interpretation of ultrasonographic features occur in up to 70% of cases (17, 19, 56). This low reproducibility, that may cause uncertainty in clinical management of thyroid nodules, emphasizes the need for an objective quantification method, such as a software device to computerize these features (54). The sensitivity and specificity of US findings vary in the literature (44, 57). The diagnostic performance of thyroid US in Bethesda category IV nodules has a lower sensitivity (50%) and positive predictive value (PPV) (50%) than that in other Bethesda categories. Thus, the importance of the current US morphological guidelines for this category is limited (58). The CAD software system, as used in this study, can reduce the difference in feature interpretation, and the reader experiences a gap while increasing reading accuracy, which may be used as an accompaniment in imaging diagnosis (20–23, 59).

We showed a difference in the quantitative index of echogenicity, calcification, echotexture, margin and tall-wide orientation in a malignant tumor with Bethesda category IV, which is different from a ‘yes’ or ‘no’ binary feature presentation. Quantitative indices provide detailed comparisons and can establish more precise predictive models in future studies.

The role of sonographic features or patterns should not only be used in selecting nodules that must be biopsied, but also in determining the management after cytological diagnosis (60). Recently, it was determined that the rate of malignancy in these types of nodules can be stratified according to sonographic patterns (60, 61). In this study, the features were used to classify using the ATA and ACR TIRAD, which are the most popular category systems to differentiate the risk of malignancy. This showed that both systems can help stratify those who may benefit from or who should have a pathological diagnosis. However, this study also showed that a relatively high percentage (21.5%, 14 of 65) of tumors had non-ATA patterns, and that there was a large gap in the malignancy rate (9.1%–51.7%) from ATA very low, low, intermediate to ATA high suspicion. In this study, the ACR TIRADS system included more feature information, and the malignant rate increased (12.5%-53.8%) with advancing CAD TIRADS scores as indicated in Table 4. This suggests that this approach may result in better stratification for Bethesda category IV tumors. Compared with the risk of malignancy of Bethesda category IV, which is highly uncertain with a range from 10% to 40%, the additional risk stratification by CAD TIRADS will further help the surgery decision. Once more studies with greater sample sizes are conducted to confirm the findings, nodules categorized with Bethesda IV and TIRADS 3 or below would be considered low risk of malignancy and may undergo continuous follow-up. Bethesda IV nodules categorized with TIRADS 4 would be considered moderate risk of malignancy and lobectomy could be suggested. A high risk of malignancy is seen in nodules categorized with Bethesda IV and TIRADS 5, and at least lobectomy or total thyroidectomy for larger nodules is suggested. Numerous variations in TIRADS have been proposed, and some of them included additional US features, such as elastography and vascularity criteria (47). The ACR TIRADS used in our study is simple, convenient, and accurate and can be used to stratify the risk of malignancy. Furthermore, we showed that 30.8% of follicular neoplasms do not present features such as heterogeneity, blurred margin, and irregular margin; their malignancy rate is 0%. These patients could be spared from surgery and monitoring. However, further large-scale studies would need to confirm the findings of the current study.

However, we found in the study that there were differences between the human observers and the CAD in ATA/TIRAD interpretation. Even among the human observers, the consistence of interpretation was also shown to be poor or fair. Thus, further large-scale studies must be conducted to confirm the findings of the current study.

This study had several limitations: first, we only included patients who underwent surgery and therefore potential selection bias may have affected the study results. Nevertheless, this bias was inevitable because histopathology was necessary to provide a reliable diagnosis in patients with Bethesda category IV nodules. Second, in order to acquire dicom-format images for software analysis, all patients enrolled in this study were arranged for additional ultrasound scans. Therefore, collecting an extremely large number of cases within a short period of time is impossible. Studies with a larger number of cases should be conducted when the CAD software can be applied more widely to routine clinical examination with a standard workflow. Third, the applicability of this study should be limited to those nodules pre-selected for FNAC based on ATA guidelines and diagnosed as Bethesda IV category. Fourth, instead of real-time images during ultrasonography, only static images selected and recorded by the US examiners were analyzed in this study. Thus, the advantage of dynamic US examination was not considered in the current study. In addition, because of the nature of this study, elastography or vascularization was not available in our workflow.

Conclusion

For patients with nodules pre-selected for FNAC based on ATA guidelines and diagnosed as Bethesda category IV, the software-based characterizations of US features, along with the associated ATA patterns and TIRADS system, were shown helpful in the risk stratification of malignancy.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by the institutional review board of the National Taiwan University Hospital approved the data collection and analyses in this study. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

M-HW, C-NC, K-YC, AC and M-SH proposed the study. M-HW, C-NC, K-YC and AC performed research, analyzed the data and wrote the first draft. M-SH collected the data. M-HW, C-NC, K-YC and AC analyzed the data. All authors contributed to the design and interpretation of the study and to further drafts. All authors contributed to the article and approved the submitted version.

Funding

The AmCad BioMed Corporation, Taipei, Taiwan, sponsored this study in terms of technical assistance and financial support.

Conflict of Interest

M-HW, K-YC, AC and C-NC received research grant support from AmCad BioMed.

The remaining author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential confict of interest.

Acknowledgments

The authors would like to thank the Department of Medical Research at the National Taiwan University Hospital for providing assistance with statistical analysis.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fendo.2021.614630/full#supplementary-material

References

1. Raber W, Kaserer K, Niederle B, Vierhapper H. Risk factors for malignancy of thyroid nodules initially identified as follicular neoplasia by fine-needle aspiration: results of a prospective study of one hundred twenty patients. Thyroid (2000) 10:709–12. doi: 10.1089/10507250050137806

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Baloch Z, LiVolsi VA, Jain P, Jain R, Aljada I, Mandel S, et al. Role of repeat fine-needle aspiration biopsy (FNAB) in the management of thyroid nodules. Diagn Cytopathol (2003) 29:203–6. doi: 10.1002/dc.10361

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Suster S. Thyroid tumors with a follicular growth pattern: problems in differential diagnosis. Arch Pathol Lab Med (2006) 130:984–8. doi: 10.5858/2006-130-984-TTWAFG

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Goldstein RE, Netterville JL, Burkey B, Johnson JE. Implications of follicular neoplasms, atypia, and lesions suspicious for malignancy diagnosed by fine-needle aspiration of thyroid nodules. Ann Surg (2002) 235:656–662; discussion 662-654. doi: 10.1097/00000658-200205000-00007

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Baloch ZW, Fleisher S, LiVolsi VA, Gupta PK. Diagnosis of “follicular neoplasm”: a gray zone in thyroid fine-needle aspiration cytology. Diagn Cytopathol (2002) 26:41–4. doi: 10.1002/dc.10043

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Cibas ES, Ali SZ, Conference NCITFSotS. The Bethesda System For Reporting Thyroid Cytopathology. Am J Clin Pathol (2009) 132:658–65. doi: 10.1309/AJCPPHLWMI3JV4LA

PubMed Abstract | CrossRef Full Text | Google Scholar

7. American Thyroid Association Guidelines Taskforce on Thyroid N, Differentiated Thyroid C, Cooper DS, Doherty GM, Haugen BR, Kloos RT, et al. Revised American Thyroid Association management guidelines for patients with thyroid nodules and differentiated thyroid cancer. Thyroid (2009) 19:1167–214. doi: 10.1089/thy.2009.0110

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Perros P, Thyroid Cancer Guidelines Update G. Introduction to the updated guidelines on the management of thyroid cancer. Clin Med (Lond) (2007) 7:321–2. doi: 10.7861/clinmedicine.7-4-321

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Musholt TJ, Clerici T, Dralle H, Frilling A, Goretzki PE, Hermann MM, et al. German Association of Endocrine Surgeons practice guidelines for the surgical treatment of benign thyroid disease. Langenbecks Arch Surg (2011) 396:639–49. doi: 10.1007/s00423-011-0774-y

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Pacini F, Burroni L, Ciuoli C, Di Cairano G, Guarino E. Management of thyroid nodules: a clinicopathological, evidence-based approach. Eur J Nucl Med Mol Imaging (2004) 31:1443–9. doi: 10.1007/s00259-004-1680-0

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Gulcelik NE, Gulcelik MA, Kuru B. Risk of malignancy in patients with follicular neoplasm: predictive value of clinical and ultrasonographic features. Arch Otolaryngol Head Neck Surg (2008) 134:1312–5. doi: 10.1001/archotol.134.12.1312

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Miyakawa M, Onoda N, Etoh M, Fukuda I, Takano K, Okamoto T, et al. Diagnosis of thyroid follicular carcinoma by the vascular pattern and velocimetric parameters using high resolution pulsed and power Doppler ultrasonography. Endocr J (2005) 52:207–12. doi: 10.1507/endocrj.52.207

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Kuo TC, Wu MH, Chen KY, Hsieh MS, Chen A, Chen CN. Ultrasonographic features for differentiating follicular thyroid carcinoma and follicular adenoma. Asian J Surg (2019) 43(1):339–46. doi: 10.1016/j.asjsur.2019.04.016

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Wienke JR, Chong WK, Fielding JR, Zou KH, Mittelstaedt CA. Sonographic features of benign thyroid nodules: interobserver reliability and overlap with malignancy. J Ultrasound Med (2003) 22:1027–31. doi: 10.7863/jum.2003.22.10.1027

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Hegedus L. Thyroid size determined by ultrasound. Influence of physiological factors and non-thyroidal disease. Dan Med Bull (1990) 37:249–63.

PubMed Abstract | Google Scholar

16. Lyshchik A, Drozd V, Schloegl S, Reiners C. Three-dimensional ultrasonography for volume measurement of thyroid nodules in children. J Ultrasound Med (2004) 23:247–54. doi: 10.7863/jum.2004.23.2.247

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Brauer VF, Eder P, Miehle K, Wiesner TD, Hasenclever H, Paschke R. Interobserver variation for ultrasound determination of thyroid nodule volumes. Thyroid (2005) 15:1169–75. doi: 10.1089/thy.2005.15.1169

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Jarlov AE, Nygard B, Hegedus L, Karstrup S, Hansen JM. Observer variation in ultrasound assessment of the thyroid gland. Br J Radiol (1993) 66:625–7. doi: 10.1259/0007-1285-66-787-625

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Choi SH, Kim EK, Kwak JY, Kim MJ, Son EJ. Interobserver and intraobserver variations in ultrasound assessment of thyroid nodules. Thyroid (2010) 20:167–72. doi: 10.1089/thy.2008.0354

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Chen KY, Chen CN, Wu MH, Ho MC, Tai HC, Huang WC, et al. Computerized detection and quantification of microcalcifications in thyroid nodules. Ultrasound Med Biol (2011) 37:870–8. doi: 10.1016/j.ultrasmedbio.2011.03.002

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Wu MH, Chen CN, Chen KY, Ho MC, Tai HC, Chung YC, et al. Quantitative analysis of dynamic power Doppler sonograms for patients with thyroid nodules. Ultrasound Med Biol (2013) 39:1543–51. doi: 10.1016/j.ultrasmedbio.2013.03.009

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Chen KY, Chen CN, Wu MH, Ho MC, Tai HC, Kuo WH, et al. Computerized quantification of ultrasonic heterogeneity in thyroid nodules. Ultrasound Med Biol (2014) 40:2581–9. doi: 10.1016/j.ultrasmedbio.2014.06.009

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Wu MH, Chen CN, Chen KY, Ho MC, Tai HC, Wang YH, et al. Quantitative analysis of echogenicity for patients with thyroid nodules. Sci Rep (2016) 6:35632. doi: 10.1038/srep35632

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Choi YJ, Baek JH, Park HS, Shim WH, Kim TY, Shong YK, et al. A Computer-Aided Diagnosis System Using Artificial Intelligence for the Diagnosis and Characterization of Thyroid Nodules on Ultrasound: Initial Clinical Assessment. Thyroid (2017) 27:546–52. doi: 10.1089/thy.2016.0372

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Jeong EY, Kim HL, Ha EJ, Park SY, Cho YJ, Han M. Computer-aided diagnosis system for thyroid nodules on ultrasonography: diagnostic performance and reproducibility based on the experience level of operators. Eur Radiol (2019) 29:1978–85. doi: 10.1007/s00330-018-5772-9

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Yoo YJ, Ha EJ, Cho YJ, Kim HL, Han M, Kang SY. Computer-Aided Diagnosis of Thyroid Nodules via Ultrasonography: Initial Clinical Experience. Korean J Radiol (2018) 19:665–72. doi: 10.3348/kjr.2018.19.4.665

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Wu MH, Chen KY, Shih SR, Ho MC, Tai HC, Chang KJ, et al. Multi-Reader Multi-Case Study for Performance Evaluation of High-Risk Thyroid Ultrasound with Computer-Aided Detection. Cancers (Basel) (2020) 12(2):373. doi: 10.3390/cancers12020373

CrossRef Full Text | Google Scholar

28. Haugen BR, Alexander EK, Bible KC, Doherty GM, Mandel SJ, Nikiforov YE, et al. 2015 American Thyroid Association Management Guidelines for Adult Patients with Thyroid Nodules and Differentiated Thyroid Cancer: The American Thyroid Association Guidelines Task Force on Thyroid Nodules and Differentiated Thyroid Cancer. Thyroid (2016) 26:1–133. doi: 10.1089/thy.2015.0020

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Tessler FN, Middleton WD, Grant EG, Hoang JK, Berland LL, Teefey SA, et al. ACR Thyroid Imaging, Reporting and Data System (TI-RADS): White Paper of the ACR TI-RADS Committee. J Am Coll Radiol (2017) 14:587–95. doi: 10.1016/j.jacr.2017.01.046

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Cibas ES, Ali SZ. The 2017 Bethesda System for Reporting Thyroid Cytopathology. Thyroid (2017) 27:1341–6. doi: 10.1089/thy.2017.0500

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Cibas ES, Ali SZ. The Bethesda System for Reporting Thyroid Cytopathology. Thyroid (2009) 19:1159–65. doi: 10.1089/thy.2009.0274

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Kuru B, Atmaca A, Tarim IA, Kefeli M, Topgul K, Yoruker S, et al. Risk factors associated with malignancy and with triage to surgery in thyroid nodules classified as Bethesda category III (AUS/FLUS). Eur J Surg Oncol (2016) 42:87–93. doi: 10.1016/j.ejso.2015.09.026

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Macias CA, Arumugam D, Arlow RL, Eng OS, Lu SE, Javidian P, et al. A risk model to determine surgical treatment in patients with thyroid nodules with indeterminate cytology. Ann Surg Oncol (2015) 22:1527–32. doi: 10.1245/s10434-014-4190-8

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Kuru B, Kefeli M. Risk factors associated with malignancy and with triage to surgery in thyroid nodules classified as Bethesda category IV (FN/SFN). Diagn Cytopathol (2018) 46:489–94. doi: 10.1002/dc.23923

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Lubitz CC, Faquin WC, Yang J, Mekel M, Gaz RD, Parangi S, et al. Clinical and cytological features predictive of malignancy in thyroid follicular neoplasms. Thyroid (2010) 20:25–31. doi: 10.1089/thy.2009.0208

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Miller B, Burkey S, Lindberg G, Snyder WH,3, Nwariaku FE. Prevalence of malignancy within cytologically indeterminate thyroid nodules. Am J Surg (2004) 188:459–62. doi: 10.1016/j.amjsurg.2004.07.006

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Kiernan CM, Solorzano CC. Bethesda Category III, IV, and V Thyroid Nodules: Can Nodule Size Help Predict Malignancy? J Am Coll Surg (2017) 225:77–82. doi: 10.1016/j.jamcollsurg.2017.02.002

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Lee SH, Baek JS, Lee JY, Lim JA, Cho SY, Lee TH, et al. Predictive factors of malignancy in thyroid nodules with a cytological diagnosis of follicular neoplasm. Endocr Pathol (2013) 24:177–83. doi: 10.1007/s12022-013-9263-x

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Krane JF, Vanderlaan PA, Faquin WC, Renshaw AA. The atypia of undetermined significance/follicular lesion of undetermined significance:malignant ratio: a proposed performance measure for reporting in The Bethesda System for thyroid cytopathology. Cancer Cytopathol (2012) 120:111–6. doi: 10.1002/cncy.20192

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Liu FH, Liou MJ, Hsueh C, Chao TC, Lin JD. Thyroid follicular neoplasm: analysis by fine needle aspiration cytology, frozen section, and histopathology. Diagn Cytopathol (2010) 38:801–5. doi: 10.1002/dc.21294

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Papini E, Guglielmi R, Bianchini A, Crescenzi A, Taccogna S, Nardi F, et al. Risk of malignancy in nonpalpable thyroid nodules: predictive value of ultrasound and color-Doppler features. J Clin Endocrinol Metab (2002) 87:1941–6. doi: 10.1210/jcem.87.5.8504

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Brito JP, Gionfriddo MR, Al Nofal A, Boehmer KR, Leppin AL, Reading C, et al. The accuracy of thyroid nodule ultrasound to predict thyroid cancer: systematic review and meta-analysis. J Clin Endocrinol Metab (2014) 99:1253–63. doi: 10.1210/jc.2013-2928

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Alexander EK, Marqusee E, Orcutt J, Benson CB, Frates MC, Doubilet PM, et al. Thyroid nodule shape and prediction of malignancy. Thyroid (2004) 14:953–8. doi: 10.1089/thy.2004.14.953

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Khoo ML, Asa SL, Witterick IJ, Freeman JL. Thyroid calcification and its association with thyroid carcinoma. Head Neck (2002) 24:651–5. doi: 10.1002/hed.10115

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Mazzaferri EL. Management of a solitary thyroid nodule. N Engl J Med (1993) 328:553–9. doi: 10.1056/NEJM199302253280807

PubMed Abstract | CrossRef Full Text | Google Scholar

46. McHenry CR, Phitayakorn R. Follicular adenoma and carcinoma of the thyroid gland. Oncologist (2011) 16:585–93. doi: 10.1634/theoncologist.2010-0405

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Kwak JY, Han KH, Yoon JH, Moon HJ, Son EJ, Park SH, et al. Thyroid imaging reporting and data system for US features of nodules: a step in establishing better stratification of cancer risk. Radiology (2011) 260:892–9. doi: 10.1148/radiol.11110206

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Kim EK, Park CS, Chung WY, Oh KK, Kim DI, Lee JT, et al. New sonographic criteria for recommending fine-needle aspiration biopsy of nonpalpable solid nodules of the thyroid. AJR Am J Roentgenol (2002) 178:687–91. doi: 10.2214/ajr.178.3.1780687

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Cronan JJ. Thyroid nodules: is it time to turn off the US machines? Radiology (2008) 247:602–4. doi: 10.1148/radiol.2473072233

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Park SH, Kim SJ, Kim EK, Kim MJ, Son EJ, Kwak JY. Interobserver agreement in assessing the sonographic and elastographic features of malignant thyroid nodules. AJR Am J Roentgenol (2009) 193:W416–423. doi: 10.2214/AJR.09.2541

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Moon WJ, Jung SL, Lee JH, Na DG, Baek JH, Lee YH, et al. Benign and malignant thyroid nodules: US differentiation–multicenter retrospective study. Radiology (2008) 247:762–70. doi: 10.1148/radiol.2473070944

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Park CS, Kim SH, Jung SL, Kang BJ, Kim JY, Choi JJ, et al. Observer variability in the sonographic evaluation of thyroid nodules. J Clin Ultrasound (2010) 38:287–93. doi: 10.1002/jcu.20689

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Valderrabano P, Klippenstein DL, Tourtelot JB, Ma Z, Thompson ZJ, Lilienfeld HS, et al. New American Thyroid Association Sonographic Patterns for Thyroid Nodules Perform Well in Medullary Thyroid Carcinoma: Institutional Experience, Systematic Review, and Meta-Analysis. Thyroid (2016) 26:1093–100. doi: 10.1089/thy.2016.0196

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Lam CA, McGettigan MJ, Thompson ZJ, Khazai L, Chung CH, Centeno BA, et al. Ultrasound characterization for thyroid nodules with indeterminate cytology: inter-observer agreement and impact of combining pattern-based and scoring-based classifications in risk stratification. Endocrine (2019) 66(2):278–87. doi: 10.1007/s12020-019-02000-0

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Slapa RZ, Slowinska-Srzednicka J, Szopinski KT, Jakubowski W. Gray-scale three-dimensional sonography of thyroid nodules: feasibility of the method and preliminary studies. Eur Radiol (2006) 16:428–36. doi: 10.1007/s00330-005-2903-x

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Hoang JK, Middleton WD, Farjat AE, Teefey SA, Abinanti N, Boschini FJ, et al. Interobserver Variability of Sonographic Features Used in the American College of Radiology Thyroid Imaging Reporting and Data System. AJR Am J Roentgenol (2018) 211:162–7. doi: 10.2214/AJR.17.19192

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Frates MC, Benson CB, Charboneau JW, Cibas ES, Clark OH, Coleman BG, et al. Management of thyroid nodules detected at US: Society of Radiologists in Ultrasound consensus conference statement. Radiology (2005) 237:794–800. doi: 10.1148/radiol.2373050220

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Park SY, Hahn SY, Shin JH, Ko EY, Oh YL. The Diagnostic Performance of Thyroid US in Each Category of the Bethesda System for Reporting Thyroid Cytopathology. PloS One (2016) 11:e0155898. doi: 10.1371/journal.pone.0155898

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Jeong EY, Kim HL, Ha EJ, Park SY, Cho YJ, Han M. Computer-aided diagnosis system for thyroid nodules on ultrasonography: diagnostic performance and reproducibility based on the experience level of operators. Eur Radiol (2018) 29(4):1978–85. doi: 10.1007/s00330-018-5772-9

PubMed Abstract | CrossRef Full Text | Google Scholar

60. Valderrabano P, McGettigan MJ, Lam CA, Khazai L, Thompson ZJ, Chung CH, et al. Thyroid Nodules with Indeterminate Cytology: Utility of the American Thyroid Association Sonographic Patterns for Cancer Risk Stratification. Thyroid (2018) 28:1004–12. doi: 10.1089/thy.2018.0085

PubMed Abstract | CrossRef Full Text | Google Scholar

61. Chng CL, Kurzawinski TR, Beale T. Value of sonographic features in predicting malignancy in thyroid nodules diagnosed as follicular neoplasm on cytology. Clin Endocrinol (Oxf) (2015) 83:711–6. doi: 10.1111/cen.12692

PubMed Abstract | CrossRef Full Text | Google Scholar