Abstract:Background: The survival rate of patients with distant metastasis (DM) of papillary thyroid carcinoma (PTC) is significantly reduced. It is of great significance to find an effective method for early prediction of the risk of DM for formulating individualized diagnosis and treatment plans and improving prognosis. Previous studies have significant limitations, and it is still necessary to develop new models for predicting the risk of DM of PTC. We aimed to develop and validate interpretable machine learning (ML) models for early prediction of DM in patients with PTC using a multicenter cohort. Methods: We collected data on patients with PTC who were admitted between June 2013 and May 2023. Data from 1430 patients at Yunnan Cancer Hospital (YCH) served as the training and internal validation set, while data from 434 patients at the First Affiliated Hospital of Kunming Medical University (KMU 1st AH) was used as the external test set. Nine ML methods such as random forest (RF) were used to construct the model. Model prediction performance was compared using evaluation indicators such as the area under the receiver operating characteristic curve (AUC). The SHapley Additive exPlanation (SHAP) method was used to rank the feature importance and explain the final model. Findings: Among the nine ML models, the RF model performed the best. The RF model accurately predicted the risk of DM in patients with PTC in both the internal validation of the training set [AUC: 0.913, 95% confidence interval (CI) (0.9075-0.9185)] and the external test set [AUC: 0.8996, 95% CI (0.8483-0.9509)]. The calibration curve showed high agreement between the predicted and observed risks. In the sensitivity analysis focusing on DM sites of PTC, the RF model exhibited outstanding performance in predicting "lung-only metastasis" showing high AUC, specificity, sensitivity, F1 score, and a low Brier score. SHAP analysis identified variables that contributed to the model predictions. An online calculator based on the RF model was developed and made available for clinicians at https://predictingdistantmetastasis.shinyapps.io/shiny1/. 11 variables were included in the final RF model: age of the patient with PTC, whether the tumor size is > 2 cm, whether the tumor size is ≤ 1 cm, lymphocyte (LYM) count, monocyte (MONO) count, monocyte/lymphocyte ratio (MLR), thyroglobulin (TG) level, thyroid peroxidase antibody (TPOAb) level, whether the T stage is T1/2, whether the T stage is T3/4, and whether the N stage is N0. Interpretation: On the basis of large-sample and multicenter data, we developed and validated an explainable ML model for predicting the risk of DM in patients with PTC. The model helps clinicians to identify high-risk patients early and provides a basis for individualized patient treatment plans. Funding: This work was supported by the National Natural Science Foundation of China (No. 81960426, 82360345 and 82001986), the Outstanding Youth Science Foundation of Yunnan Basic Research Project (No. 202401AY070001-316), Yunnan Province Applied and Basic Research Foundation (No. 202401AT070008), and Ten Thousand Talent Plans for Young Top-notch Talents of Yunnan Province.

Development and validation of prediction models for papillary thyroid cancer structural recurrence using machine learning approaches

Deep Learning-based Risk Prediction Model for Recurrence-free Survival in Patients with Hepatocellular Carcinoma Using Multi-phase CT Image

Machine learning-based prediction model using clinico-pathologic factors for papillary thyroid carcinoma recurrence

Nomogram-Based New Recurrence Predicting System in Early-Stage Papillary Thyroid Cancer

Improving the Risk Prediction of the 2015 ATA Recurrence Risk Stratification in Papillary Thyroid Cancer

Development and validation of an interpretable machine learning model for predicting the risk of distant metastasis in papillary thyroid cancer: a multicenter study

An ultrasound model for predicting recurrence of papillary thyroid carcinoma after complete endoscopic resection

The innovative model based on artificial intelligence algorithms to predict recurrence risk of patients with postoperative breast cancer

An individualized protein-based prognostic model to stratify pediatric patients with papillary thyroid carcinoma

Prediction Model of Cervical Lymph Node Metastasis in Papillary Thyroid Carcinoma

Development of a Machine Learning Model to Predict Recurrence of Oral Tongue Squamous Cell Carcinoma

A comparative analysis of eight machine learning models for the prediction of lateral lymph node metastasis in patients with papillary thyroid carcinoma

Machine learning for risk stratification of thyroid cancer patients: a 15-year cohort study

Developing and validating a multivariable machine learning model for the preoperative prediction of lateral lymph node metastasis of papillary thyroid cancer

Machine learning-based models for the prediction of breast cancer recurrence risk

Prediction of cervical lymph node metastasis in solitary papillary thyroid carcinoma based on ultrasound radiomics analysis

Machine learning-based dynamic prediction of lateral lymph node metastasis in patients with papillary thyroid cancer

Risk stratification of papillary thyroid cancers using multidimensional machine learning

A hybrid machine learning model combining association rule mining and classification algorithms to predict differentiated thyroid cancer recurrence

Machine Learning Algorithms to Predict Recurrence within 10 Years after Breast Cancer Surgery: A Prospective Cohort Study

mtPCDI: a machine learning-based prognostic model for prostate cancer recurrence