Abstract:Diagnosis and treatment of follicular thyroid neoplasms (FTN) remains a huge challenge, as the malignancy risk of FTN cannot be ascertained until the diagnostic surgery. We aimed to use interpretable machine learning to predict the malignant risk of FTN before surgery in the real-world setting. We conducted a retrospective cohort study at the Peking University Third Hospital in Beijing, China between January 2012 and September 2023. We included patients with postoperative pathological diagnoses of follicular thyroid adenoma (FTA) or carcinoma (FTC) and excluded those without thyroid ultrasonography before surgery. We used 22 predictors of demographic characteristics, thyroid sonography, and hormones to train 5 machine learning models and chose the optimal model that performed well in model discrimination, calibration, interpretability, and parsimony. For this extremely unbalanced data (the ratio of FTA to FTC much greater than 1), we adopted both the area under receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC) to assess model discrimination. We also summarized the experiences derived from our study findings and a literature review of the existing evidence. This cohort included 1543 patients (mean age: 47.98 ± 14.14 years, female: 73%) with 1676 FTN tumors (FTA: n = 1418; FTC: n = 258; the ratio of FTA to FTC: 5.5). The random forest was chosen as the optimal model, revealing that mean TSH score, mean tumor diameter, mean TSH, margin, and TSH instability were the five most important predictors in discriminating FTA from FTC, with the AUROC of 0.79 (95% CI, 0.77-0.80) and AUPRC of 0.40 (0.38-0.42). The effect of mean diameter on malignancy risk varied with TSH instability; specifically, the malignancy risk was elevated in larger tumors with greater TSH instability, whereas it was reduced in larger tumors with more stable TSH levels. The risk of malignancy tended to nonlinearly increase with a larger mean tumor diameter or higher instability in TSH levels, while it nonlinearly decreased with a higher mean TSH score or mean TSH level. FTCs with small size (mean diameter: 2.89 cm) were more likely to be misclassified as FTAs than those with greater size. Our literature review showed that (1) the ratio of FTA to FTC varied from 0.6 to 4.0 which was lower than the natural distribution of 5.0; (2) no studies have used the AUPRC to assess prediction performance in the unbalanced data; and (3) the external validation of previously derived model did not perform as well as in the original study. TSH measurements and tumor size were important in screening the malignancy risk of FTN before the diagnostic surgery. However, it remained challenging to assess the malignancy risk of small-size FTNs. Future studies are needed to overcome the prediction challenge embedded in the extremely unbalanced distribution of FTA and FTC in real-world data. This study was approved by the Medical Research Ethics Committee of Peking University Third Hospital (No. IRB00006761-M2023168).

A Multimodal Deep Learning Model for Preoperative Risk Prediction of Follicular Thyroid Carcinoma

Deep Fusion Models of Multi-Phase CT and Selected Clinical Data for Preoperative Prediction of Early Recurrence in Hepatocellular Carcinoma.

A Transformer-Based Model for Preoperative Early Recurrence Prediction of Hepatocellular Carcinoma with Muti-modality MRI

A Unified Framework for Preoperative Early Recurrence Prediction of Hepatocellular Carcinoma with Multi-phase CT Images

Deep Learning-Based Radiomics Models for Early Recurrence Prediction of Hepatocellular Carcinoma with Multi-phase CT Images and Clinical Data.

Model Based on Ultrasound Radiomics and Machine Learning to Preoperative Differentiation of Follicular Thyroid Neoplasm

Multimodal Diverse Granularity Fusion Network based on US and CT Images for Lymph Node Metastasis Prediction of Thyroid Carcinoma

Multi-modal ultrasound multistage classification of PTC cervical lymph node metastasis via DualSwinThyroid

Automated diagnosis and management of follicular thyroid nodules based on the devised small-dataset interpretable foreground optimization network deep learning: a multicenter diagnostic study

Development and validation of a predictive model for assessing the risk of follicular carcinoma in thyroid nodules identified as suspicious by intraoperative frozen section

Predicting Malignancy in Thyroid Nodules Based on Conventional Ultrasound and Elastography: the Value of Predictive Models in a Multi-Center Study

DeepThy‐Net: A Multimodal Deep Learning Method for Predicting Cervical Lymph Node Metastasis in Papillary Thyroid Cancer

A model for predicting lymph node metastasis of thyroid carcinoma: a multimodality convolutional neural network study

A proof-of-concept investigation into predicting follicular carcinoma on ultrasound using topological data analysis and radiomics

Multimodal MRI Deep Learning for Predicting Central Lymph Node Metastasis in Papillary Thyroid Cancer

Ultrasound radiomics models based on multimodal imaging feature fusion of papillary thyroid carcinoma for predicting central lymph node metastasis

Automatic tumor segmentation and lymph node metastasis prediction in papillary thyroid carcinoma using ultrasound keyframes

Diagnostic value of an interpretable machine learning model based on clinical ultrasound features for follicular thyroid carcinoma

Interpretable Machine Learning to Predict the Malignancy Risk of Follicular Thyroid Neoplasms in the Extremely Unbalanced Data: Experiences from a Real-World Study and Literature Review (Preprint)

Multiparametric magnetic resonance imaging in preoperative assessment of follicular thyroid neoplasm

Preoperative Molecular Subtype Classification Prediction of Ovarian Cancer Based on Multi-Parametric Magnetic Resonance Imaging Multi-Sequence Feature Fusion Network