Abstract:Objectives: To apply machine learning to extract radiomics features from thyroid two-dimensional ultrasound (2D-US) combined with contrast-enhanced ultrasound (CEUS) images to classify and predict benign and malignant thyroid nodules, classified according to the Chinese version of the thyroid imaging reporting and data system (C-TIRADS) as category 4. Materials and methods: This retrospective study included 313 pathologically diagnosed thyroid nodules (203 malignant and 110 benign). Two 2D-US images and five CEUS key frames ("2nd second after the arrival time" frame, "time to peak" frame, "2nd second after peak" frame, "first-flash" frame, and "second-flash" frame) were selected to manually label the region of interest using the "Labelme" tool. A total of 7 images of each nodule and their annotates were imported into the Darwin Research Platform for radiomics analysis. The datasets were randomly split into training and test cohorts in a 9:1 ratio. Six classifiers, namely, support vector machine, logistic regression, decision tree, random forest (RF), gradient boosting decision tree and extreme gradient boosting, were used to construct and test the models. Performance was evaluated using a receiver operating characteristic curve analysis. The area under the curve (AUC), sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), accuracy (ACC), and F1-score were calculated. One junior radiologist and one senior radiologist reviewed the 2D-US image and CEUS videos of each nodule and made a diagnosis. We then compared their AUC and ACC with those of our best model. Results: The AUC of the diagnosis of US, CEUS and US combined CEUS by junior radiologist and senior radiologist were 0.755, 0.750, 0.784, 0.800, 0.873, 0.890, respectively. The RF classifier performed better than the other five, with an AUC of 1 for the training cohort and 0.94 (95% confidence interval 0.88-1) for the test cohort. The sensitivity, specificity, accuracy, PPV, NPV, and F1-score of the RF model in the test cohort were 0.82, 0.93, 0.90, 0.85, 0.92, and 0.84, respectively. The RF model with 2D-US combined with CEUS key frames achieved equivalent performance as the senior radiologist (AUC: 0.94 vs. 0.92, P = 0.798; ACC: 0.90 vs. 0.92) and outperformed the junior radiologist (AUC: 0.94 vs. 0.80, P = 0.039, ACC: 0.90 vs. 0.81) in the test cohort. Conclusions: Our model, based on 2D-US and CEUS key frames radiomics features, had good diagnostic efficacy for thyroid nodules, which are classified as C-TIRADS 4. It shows promising potential in assisting less experienced junior radiologists.

Efficacy of color Doppler ultrasound signs combined with serum tumor-specific growth factor in the diagnosis of differentiated thyroid cancer

Ultrasound-based radiomics nomogram combined with clinical features for the prediction of central lymph node metastasis in papillary thyroid carcinoma patients with Hashimoto's thyroiditis

Diagnostic value of an interpretable machine learning model based on clinical ultrasound features for follicular thyroid carcinoma

Predicting Malignancy in Thyroid Nodules Based on Conventional Ultrasound and Elastography: the Value of Predictive Models in a Multi-Center Study

Model Based on Ultrasound Radiomics and Machine Learning to Preoperative Differentiation of Follicular Thyroid Neoplasm

Predicting Malignancy in Thyroid Nodules with Benign Cytology Results: the Role of Conventional Ultrasound, Shear Wave Elastography and BRAF V600E

Combined Fine-Needle Aspiration and Selective Intraoperative Frozen Section to Optimize Prediction of Malignant Thyroid Nodules: A Retrospective Cohort Study of More Than 3000 Patients

Diagnostic Value of Contrast-Enhanced Ultrasound Combined with Fine-Needle Aspiration for Thyroid Cancer

Development and Validation of a Novel Diagnostic Tool for Predicting the Malignancy Probability of Thyroid Nodules: A Retrospective Study Based on Clinical, B-mode, Color Doppler and Elastographic Ultrasonographic Characteristics.

Application of Ultrasound Model for Predicting Central Compartment Lymph Node Metastasis in Papillary Thyroid Carcinoma

Diagnostic value of qualitative and quantitative parameters of contrast-enhanced ultrasound for differentiating differentiated thyroid carcinomas from benign nodules

Value of Combining Clinical Factors, Conventional Ultrasound, and Contrast-Enhanced Ultrasound Features in Preoperative Prediction of Central Lymph Node Metastases of Different Sized Papillary Thyroid Carcinomas

Incorporation of CEUS and SWE Parameters into a Multivariate Logistic Regression Model for the Differential Diagnosis of Benign and Malignant TI-RADS4 Thyroid Nodules

Predicting tall-cell subtype of papillary thyroid carcinomas independently with preoperative multimodal ultrasound.

Diagnostic value of color doppler ultrasound for papillary thyroid carcinoma

Development and validation of a predictive model for assessing the risk of follicular carcinoma in thyroid nodules identified as suspicious by intraoperative frozen section

Predictive Value of the Nomogram Model Based on Multimodal Ultrasound Features for Benign and Malignant Thyroid Nodules of C-TIRADS Category 4

Development and validation of the nomogram based on ultrasound, thyroid stimulating hormone, and inflammatory marker in papillary thyroid carcinoma: a case-control study

Evaluation of the value of combined thyroid function-related indexes in the prognosis prediction of patients with differentiated thyroid cancer

Applying machine-learning models to differentiate benign and malignant thyroid nodules classified as C-TIRADS 4 based on 2D-ultrasound combined with five contrast-enhanced ultrasound key frames

Prediction of cervical lymph node metastasis in solitary papillary thyroid carcinoma based on ultrasound radiomics analysis