Journals | Policy | Permission

Journal of Endocrinology & Metabolism

Original Article

Volume 11, Number 3-4, August 2021, pages 69-75

Diagnostic Performance of ACR-TIRADS in Differentiating Benign From Malignant Thyroid Nodules in Patients Undergoing Fine-Needle Aspiration Biopsy: Comparative Study Based on Five International Guidelines for Management of Thyroid Nodules

Thyroid nodules are commonly seen in everyday practice. The main question we want to answer is whether the nodule is benign or malignant and with what certainty. High-resolution ultrasound is the most effective way of evaluating thyroid nodules [1]. Many international risk stratification systems are used throughout the world to evaluate and guide management of thyroid nodules, guiding further work-up with fine-needle aspiration cytology (FNAC). The notable guidelines for evaluation of thyroid nodules are American College of Radiology Thyroid Imaging Reporting and Data System (ACR-TIRADS), American Thyroid Association (ATA), European Thyroid Imaging and Reporting Data System (EU-TIRADS), Korean Society of Thyroid Radiology (KSThR) and Kwak-TIRADS (K-TIRADS) [2-4].

The study was approved by Institutional Review Board at Yale New Haven Health Bridgeport Hospital, Connecticut, USA and St. Vincent’s Hospital at Hartford Healthcare, Connecticut, USA. All procedures performed in the studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

This study is a multi-institution retrospective study conducted from 2016 to 2020 that evaluated 200 consecutive biopsied thyroid nodules. The nodules were reclassified according to ACR-TIRADS, K-TIRADS, KSThR, EU-TIRADS, and ATA guidelines by six radiologists with 10 - 30 years of experience. The nodule characteristics were recorded and classified according to size, shape, margin, orientation, echotexture, echogenicity, presence of microcalcifications, rim or coarse calcification. A Chi-square test and multiple regression analysis were performed to calculate sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).

The institutional database was reviewed. The study population comprised consecutive 432 patients comprising 200 nodules who underwent thyroid nodule biopsy, 100 at each semi-academic and private community hospitals.

The ultrasound (US) examinations were performed by radiologic technicians with more than 10 years of experience. The US units utilized 5 - 18 MHz linear array transducers. All the images were reviewed with six radiologists with 10 - 30 years of experience. The radiologists were blinded to the pathology results. All biopsies were performed with a 25-G needle, at least three passes, under direct US guidance by experienced radiologist. The biopsies were sent on slides and fixed with cytofix/cytoperm solution (BD Biosciences) fixation buffer medium containing paraformaldehyde.

The US images were categorized according to the size, composition (cystic, spongiform, mixed, solid), echogenicity (anechoic, hyperechoic or isoechoic, hypoechoic, very hypoechoic), peripheral halo (present, absent), shape/orientation (wide, tall), margins (smooth, ill-defined, lobulated or irregular, extrathyroidal extension), echogenic foci (none or comet tail, macrocalcifications, peripheral, punctate), calcifications (large, rim, punctate) and lymph node metastasis (abnormal, normal). The vascularity was not included in the criteria. The FNA results were then analyzed and compared to the recommendations. The cytopathologic diagnosis was classified as benign or malignant. Sensitivity, specificity, PPV, NPV, risk estimates (exact 95% confidence intervals (CIs) and Pearson’s Chi-square test) of each of the five guidelines compared to the biopsy results were calculated. The unnecessary biopsy rate was defined as the percentage of benign nodules among total biopsy-required nodules. Logistic regression (outcome = malignant diagnosis) was used to generate receiver operating curves (ROCs); the area under the curve (AUC) was used to determine which test performed the best compared to the biopsy. False negative rates and unnecessary biopsy rates were calculated. The statistical analysis was performed on Statistical Analytical Software (SAS) edition V9.4.

A total of 200 nodules were included in this study. Patients ranged from 23 to 74 years of age including 36 males and 164 females. The female/male ratio was 4.5:1. Female predominance was seen among most of the age groups. The cohort showed 26 (13%) malignant nodules and 174 (87%) benign nodules. Maximum number of malignant nodules (3%) was found in 30 - 40 years age group. There were 26 histologically proven malignant nodules and 174 histologically proven benign nodules. Of the 26 malignant nodules, five were follicular carcinomas with Hurthle cell oncocytic type, 18 were papillary carcinoma and three were atypical cells of indeterminate significance. The nodules with indeterminate cytology were excluded from the study. A solid component was observed in the majority of malignant nodules (61.6%, P = 0.0376) and mixed component was observed in the majority of benign nodules (51.7%, P = 0.0376) (Table 1). There was no statistically significant difference in differentiating benign from malignant nodule with the echogenicity or orientation of the nodule. The statistically significant features of a benign nodule were spongiform appearance, no echogenic foci or comet tail and absence of peripheral halo (P < 0.03). As only biopsied nodules were included in the study, the cystic features were excluded. The statistically significant features of a malignant nodule were a solid, peripheral halo, peripheral or punctate echogenic foci, microcalcification, and macrocalcification (P < 0.001) (Table 1).

Click to view

Table 1. Nodule Characteristics

The comparative analysis of the different guidelines shows that all the classification guidelines have increased predictability of malignancy as we go higher in the grading system (Table 2). The risk of malignancy for ACR-TIRADS in our study was 0% for TR2, 3.1% for TR3, 14.3% for TR4 and 41.7% for TR5. The risk of malignancy for ATA in our study was 0% for very low, 3.7% for low, 13% for intermediate and 39% for high. The risk of malignancy for K-TIRADS in our study was 0% for 3 - probably benign, 2.9% for 4a - low suspicion, 12.8% for 4b - intermediate suspicion and 54.5% for 4c - moderate suspicion. The risk of malignancy for EU-TIRADS in our study was 0% for benign, 0% for low, 6.3% for intermediate and 32.1% for high. The risk of malignancy for KSThR in our study was 0% for TR2, 0% for TR3, 8.5% for TR4 and 33.3% for TR5.

Click to view

Table 2. Relation Between Pathological Type and the Classification Systems

The ACR-TIRADS had the highest specificity (40.23% (95% CI 32 - 47)), PPV (18.75 ((95% CI 0.12 - 0.26)), NPV (97.22 (95% CI 0.90 - 0.99)) and AUC (0.6627 (95% CI 0.59 - 0.72)), and lowest unnecessary biopsy rate was highest too (104 (52%) (P = 0.0013)). This was closely followed by ATA which demonstrated the PPV of 17.39 (95% CI 0.11 - 0.24), NPV of 96.77 (95% CI 0.89 - 0.99) and AUC of 0.6340 (95% CI 0.57 - 0.69). The K-TIRADS had the highest sensitivity (96.15% (95% CI 80 - 99)) and highest unnecessary biopsy rate (160 (80%) (P = 0.4482)) (Table 3). Our study had 37 sub-centimeter nodules that were biopsied. ACR-TIRADS had the highest correlation between sensitivity and specificity depicted with largest AUC in the ROC analysis closely followed by ATA guidelines (Fig. 1). We found that the false negative rates were lowest for K-TIRADS (0.5%) closely followed by ACR-TIRADS (1%) and ATA (1%) and highest for EU-TIRADS (3%) (Table 3). Lowest unnecessary biopsy rates were found with ACR-TIRADS (104 (52%) (P = 0.0013)) and KSThR guidelines (114 (57%) (P = 0.0059)) and highest with K-TIRADS (160 (80%) (P = 0.4482)) (Table 3).

Click to view

Table 3. Diagnostic Comparison of the Five Classification Systems

Click for large image

Figure 1. ROC analysis of guidelines. Note that ACR-TIRADS (blue line) has the highest correlation between sensitivity and specificity depicted with largest AUC in the ROC analysis closely followed by ATA guidelines (dotted red lines). ROC: receiver operating curves; ACR-TIRADS: American College of Radiology Thyroid Imaging Reporting and Data System; ATA: American Thyroid Association; EU-TIRADS: European Thyroid Imaging and Reporting Data System; KSThR: Korean Society of Thyroid Radiology; K-TIRADS: Kwak-TIRADS; AUC: area under the curve.

With high-resolution US, there is increased detection of thyroid nodules resulting in increased thyroid biopsies performed and reported higher cancer detection rates [5, 6]. TIRADS was designed in 2017 with the primary goal of better risk stratification and reduction in biopsy rates [4]. Overdiagnosis of thyroid cancer cases accounts for 70-80% cases in women and 45% in men in the USA and in many other countries [7].

Kim et al in 2002 suggested not to use size criteria and recommended biopsy of incidentally detected thyroid nodules even if one of the suspicious features was present. They described microcalcifications, irregular or microlobulated margin, markedly hypoechoic and taller than wide features as suspicious [8]. In 2011, Kwak et al categorized thyroid nodules in a similar fashion to BIRADS into six categories based on the suspicious features like solid components, hypoechogenicity especially markedly hypoechoic, microcalcifications, microlobulated or irregular margins and taller than wide for nodules more than 1 cm. The K-TIRADS was classified as K-TIRADS 1 (negative), K-TIRADS 2 (benign), K-TIRADS 3 (probably benign - no suspicious features), K-TIRADS 4A (low risk of malignancy - one suspicious feature), K-TIRADS 4B (intermediate risk of malignancy - two suspicious features), K-TIRADS 4C (moderate risk of malignancy - three or four suspicious features), K-TIRADS 5 (highly suggestive of malignancy - five suspicious features), and K-TIRADS 6 (biopsy proven malignancy) [9, 10].

ACR-TIRADS categorizes the nodules according to their composition, echogenicity, shape, margin and echogenic foci and then adding up the points. The ACR-TIRAD category is according to the points and is categorized as TR1 (0 point - benign), TR2 (2 points - not suspicious), TR3 (3 points - mildly suspicious), TR4 (points 4-6 - moderately suspicious) or TR5 (points more than 7 - highly suspicious). The size cutoff for recommended biopsy for suspicious nodules is more than 1 cm and for low-risk nodules is more than 2.5 cm [11].

EU-TIRADS categorizes nodules according to pattern recognition into benign and low-, intermediate-, and high-risk nodules, as well as recommendations for FNA. Along with the general pattern recognition, EU-TIRADS takes into account the peripheral halo, vascularity and elastography. The size cutoff for biopsy of high-risk nodules is less than 1 cm and of low-risk nodules is more than 2 cm [3].

KSThR categorizes the thyroid nodules, risk stratification and recommendation for FNA along with stratification and indications for lymph node FNA. KSThR classifies thyroid US into five categories from TR1 (no nodule), TR2 (benign), TR3 (low probability), TR3 (intermediate probability) and TR4 (high probability). The size cutoff for recommended biopsy for suspicious nodules is more than 1 cm and for low-risk nodules is more than 2 cm. KSThR also suggests that FNA may be considered in suspicious nodules more than 0.5 cm in young or middle-aged adults by shared decision making and recommended biopsy of suspicious nodes with short axis diameter of 3 - 5 mm and of intermediate nodules with short axis diameter of > 5 mm. KSThR results in higher sampling rates due to lower threshold for biopsy and thus, higher cancer detection rates [12-14].

ATA recommends survey of cervical lymph nodes for all cases with thyroid nodules. ATA categorizes the nodules into benign, very low, low, intermediate and high suspicion based on the US characteristics. The size cutoff for recommended biopsy for suspicious nodules is more than 1 cm and for low-risk nodules is more than 2 cm. They also suggested elastograpgy for preoperative risk assessment where available [15].

Thus, according to the size criteria, ACR-TIRADS recommends FNA biopsy of suspicious nodules > 1 cm and for low-risk nodules > 2.5 cm, the ATA recommends FNA for nodules >1 cm in size if there are suspicious features and > 0.5 cm if the patient has high-risk factors and for low-risk nodules > 2 cm, the K-TIRADS recommends FNA for suspicious nodules > 1 cm, for low-risk nodules > 2 cm and for suspicious nodules > 0.5 cm in young or middle-aged adults by shared decision making, KSThR recommends biopsy for high-risk nodules > 1 cm, solid nodules > 1.5 cm and for low-risk nodules > 2 cm, and EU-TIRADS recommends biopsy for high-risk nodules > 1 cm and for low-risk nodules > 2 cm [3, 11, 14, 16] (Table 4).

Click to view

Table 4. Size Criteria for Nodule Biopsy

In our study, the prevalence of thyroid cancer is 13% which is similar to worldwide incidence, thus our cohort was representative of the population [17]. Our study found that the ACR-TIRADS has the highest specificity, PPV and NPV which is in line with previous published data. ACR-TIRADS has the highest correlation between sensitivity and specificity depicted with largest AUC in the ROC analysis. The false negative rates were lowest for K-TIRADS (0.5%) closely followed by ACR-TIRADS (1%) and ATA (1%) and highest for EU-TIRADS (3%) which is in line with previous studies [18-20]. As the K-TIRADS recommends lower size cutoff of 0.5 cm, that might be the reason for higher sensitivity in our study. The unnecessary biopsy rates were highest with K-TIRADS and KSThR and lowest with ACR-TIRADS which is similar to previously published data, and again likely due to the lower size threshold with K-TIRADS and KSThR [17, 19, 21].

The risk of malignancy for ACR-TIRADS in our study was 0% for TR2, 3.1% for TR3, 14.3% for TR4 and 41.7% for TR5 which is similar to the ACR-TIRADS risk estimates [4, 20]. The risk of malignancy for K-TIRADS in our study was 0% for 3 - probably benign, 2.9% for 4a - low suspicion, 12.8% for 4b - intermediate suspicion and 54.5% for 4c - moderate suspicion similar to prior studies. The risk of malignancy for EU-TIRADS in our study was 0% for benign, 0% for low, 6.3% for intermediate and 32.1% for high, which is similar to prior studies [3]. The risk of malignancy in our study for ATA, and KSThR was similar in low/intermediate suspicion group but falls short of very high estimated risk in TR5/high suspicion group [14, 20]. This could be due to our small sample size.

One of the strengths of this study is that it shows the reproducibility of ACR-TIRADS to community level settings and with varied reader experience. Also, adopting ACR-TIRADS leads to reduction in the number of unnecessary biopsies for benign lesions. Another strength of our study is moderate sample size reflective of usual diverse population in two community hospitals that are evaluated for thyroid pathologies encountered in routine clinical practice.

There are some limitations of our study. Firstly, ours is a retrospective design, and we only included nodules which were biopsied. Secondly, the definitions of various classification systems have changed over time and there is possibility of overlap in nomenclature that might lead to different assessment during initial examination. Thirdly, nodule characteristics were determined independently by blinded readers, so it remains possible that there was mischaracterization error during US evaluation of the nodules. Lastly, the thyroid nodules were diagnosed on the basis of cytological analysis alone, not the surgical pathology.

ACR-TIRADS and ATA classification are reliable, noninvasive, and practical methods for assessing thyroid nodules in clinical practice. Both these guidelines can avoid unnecessary biopsies in a significant proportion of benign thyroid lesions. ACR-TIRADS showed highest specificity and has the highest correlation between sensitivity and specificity in identifying malignant lesions.

Acknowledgments

None to declare.

Financial Disclosure

No funding was received for this study.

Conflict of Interest

The authors declare that they have no conflict of interest.

Informed Consent

Informed consent was obtained where relevant.

Author Contributions

Guarantor of integrity of the entire study: Pranav Sharma and Steven Cohen. Study concepts and design: Pranav Sharma, Kareem Elfatairy and Darshan Gandhi. Literature research: Pranav Sharma. Clinical studies and data collection: Pranav Sharma, Kareem Elfatairy, Darshan Gandhi and Harpreet Sawhney. Experimental studies/data analysis: Pranav Sharma and Kareem Elfatairy. Statistical analysis: Pranav Sharma and Kareem Elfatairy. Manuscript preparation: Pranav Sharma. Manuscript editing: Pranav Sharma and Puneet S. Kochar.

Data Availability

The authors declare that data supporting the findings of this study are available within the article.

1. Hoang JK, Lee WK, Lee M, Johnson D, Farrell S. US Features of thyroid malignancy: pearls and pitfalls. Radiographics. 2007;27(3):847-860; discussion 861-845.
   doi pubmed
2. Middleton WD, Teefey SA, Reading CC, Langer JE, Beland MD, Szabunio MM, Desser TS. Comparison of performance characteristics of American college of radiology TI-RADS, Korean society of thyroid radiology TIRADS, and American Thyroid Association guidelines. AJR Am J Roentgenol. 2018;210(5):1148-1154.
   doi pubmed
3. Russ G, Bonnema SJ, Erdogan MF, Durante C, Ngu R, Leenhardt L. European thyroid association guidelines for ultrasound malignancy risk stratification of thyroid nodules in adults: the EU-TIRADS. Eur Thyroid J. 2017;6(5):225-237.
   doi pubmed
4. Tessler FN, Middleton WD, Grant EG, Hoang JK, Berland LL, Teefey SA, Cronan JJ, et al. ACR thyroid imaging, reporting and data system (TI-RADS): white paper of the ACR TI-RADS committee. J Am Coll Radiol. 2017;14(5):587-595.
   doi pubmed
5. American Cancer Society: Cancer Facts and Figures 2019. Atlanta, Ga: American Cancer Society. 2019.
6. Fisher SB, Perrier ND. The incidental thyroid nodule. CA Cancer J Clin. 2018;68(2):97-105.
   doi pubmed
7. Vaccarella S, Franceschi S, Bray F, Wild CP, Plummer M, Dal Maso L. Worldwide thyroid-cancer epidemic? The increasing impact of overdiagnosis. N Engl J Med. 2016;375(7):614-617.
   doi pubmed
8. Kim EK, Park CS, Chung WY, Oh KK, Kim DI, Lee JT, Yoo HS. New sonographic criteria for recommending fine-needle aspiration biopsy of nonpalpable solid nodules of the thyroid. AJR Am J Roentgenol. 2002;178(3):687-691.
   doi pubmed
9. Yoon JH, Han K, Kim EK, Moon HJ, Kwak JY. Diagnosis and management of small thyroid nodules: a comparative study with six guidelines for thyroid nodules. Radiology. 2017;283(2):560-569.
   doi pubmed
10. Kwak JY, Han KH, Yoon JH, Moon HJ, Son EJ, Park SH, Jung HK, et al. Thyroid imaging reporting and data system for US features of nodules: a step in establishing better stratification of cancer risk. Radiology. 2011;260(3):892-899.
    doi pubmed
11. Tessler FN, Middleton WD, Grant EG. Thyroid imaging reporting and data system (TI-RADS): a user's guide. Radiology. 2018;287(1):29-36.
    doi pubmed
12. Ha EJ, Na DG, Baek JH, Sung JY, Kim JH, Kang SY. US fine-needle aspiration biopsy for thyroid malignancy: diagnostic performance of seven society guidelines applied to 2000 thyroid nodules. Radiology. 2018;287(3):893-900.
    doi pubmed
13. Hong HS, Lee JY. Diagnostic Performance of Ultrasound Patterns by K-TIRADS and 2015 ATA guidelines in risk stratification of thyroid nodules and follicular lesions of undetermined significance. AJR Am J Roentgenol. 2019;213(2):444-450.
    doi pubmed
14. Shin JH, Baek JH, Chung J, Ha EJ, Kim JH, Lee YH, Lim HK, et al. Ultrasonography diagnosis and imaging-based management of thyroid nodules: revised Korean society of thyroid radiology consensus statement and recommendations. Korean J Radiol. 2016;17(3):370-395.
    doi pubmed
15. Haugen BR, Alexander EK, Bible KC, Doherty GM, Mandel SJ, Nikiforov YE, Pacini F, et al. 2015 American Thyroid Association Management Guidelines for adult patients with thyroid nodules and differentiated thyroid cancer: the American Thyroid Association guidelines task force on thyroid nodules and differentiated thyroid cancer. Thyroid. 2016;26(1):1-133.
    doi pubmed
16. Gharib H, Papini E, Valcavi R, Baskin HJ, Crescenzi A, Dottorini ME, Duick DS, et al. American Association of Clinical Endocrinologists and Associazione Medici Endocrinologi medical guidelines for clinical practice for the diagnosis and management of thyroid nodules. Endocr Pract. 2006;12(1):63-102.
    doi pubmed
17. Yoon JH, Lee HS, Kim EK, Moon HJ, Kwak JY. Malignancy risk stratification of thyroid nodules: comparison between the thyroid imaging reporting and data system and the 2014 American Thyroid Association Management Guidelines. Radiology. 2016;278(3):917-924.
    doi pubmed
18. Ha SM, Baek JH, Na DG, Suh CH, Chung SR, Choi YJ, Lee JH. Diagnostic performance of practice guidelines for thyroid nodules: thyroid nodule size versus biopsy rates. Radiology. 2019;291(1):92-99.
    doi pubmed
19. Grani G, Lamartina L, Ascoli V, Bosco D, Biffoni M, Giacomelli L, Maranghi M, et al. Reducing the number of unnecessary thyroid biopsies while improving diagnostic accuracy: toward the "Right" TIRADS. J Clin Endocrinol Metab. 2019;104(1):95-102.
    doi pubmed
20. Wu XL, Du JR, Wang H, Jin CX, Sui GQ, Yang DY, Lin YQ, et al. Comparison and preliminary discussion of the reasons for the differences in diagnostic performance and unnecessary FNA biopsies between the ACR TIRADS and 2015 ATA guidelines. Endocrine. 2019;65(1):121-131.
    doi pubmed
21. Thattarakkal VR, Ahmed TSF, Saravanam PK, et al. Evaluation of thyroid nodule: thyroid imaging reporting and data system (TIRADS) and clinicopathological correlation. Indian J Otolaryngol Head Neck Surg. 2021.
    doi

This article is distributed under the terms of the Creative Commons Attribution Non-Commercial 4.0 International License, which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Journal of Endocrinology and Metabolism is published by Elmer Press Inc.