Abstract:Background: Early preoperative evaluation of cervical lymph node metastasis (LNM) in papillary thyroid carcinoma (PTC) is critical for further surgical treatment. However, insufficient accuracy in predicting LNM status for PTC based on ultrasound images is a problem that needs to be urgently resolved. This study aimed to clarify the role of convolutional neural networks (CNNs) in predicting LNM for PTC based on multimodality ultrasound. Methods: In this study, the data of 308 patients who were clinically diagnosed with PTC and had confirmed LNM status via postoperative pathology at Beijing Tiantan Hospital, Capital Medical University, from August 2018 to April 2022 were incorporated into CNN algorithm development and evaluation. Of these patients, 80% were randomly included into the training set and 20% into the test set. The ultrasound examination of cervical LNM was performed to assess possible metastasis. Residual network 50 (Resnet50) was employed for feature extraction from the B-mode and contrast-enhanced ultrasound (CEUS) images. For each case, all of features were extracted from B-mode ultrasound images and CEUS images separately, and the ultrasound examination data of cervical LNM information were concatenated together to produce a final multimodality LNM prediction. Sensitivity, specificity, accuracy, and area under the receiver operating characteristic curve (AUC) were used to evaluate the performance of the predictive model. Heatmaps were further developed for visualizing the attention region of the images of the best-working model. Results: Of the 308 patients with PTC included in the analysis, 158 (51.3%) were diagnosed as LNM and 150 (48.7%) as non-LNM. In the test set, when a triple-modality method (i.e., B-mode image, CEUS image, and ultrasound examination of cervical LNM) was used, accuracy was maximized at 80.65% (AUC =0.831; sensitivity =80.65%; specificity =82.26%), which showed an expected increased performance over B-mode alone (accuracy =69.00%; AUC =0.720; sensitivity =70.00%; specificity =73.00%) and a dual-modality method (B-mode image plus CEUS image: accuracy =75.81%; AUC =0.742; sensitivity =74.19%; specificity =77.42%). The heatmaps of our triple-modality model demonstrated a possible focus area and revealed the model's flaws. Conclusions: The PTC lymph node prediction model based on the triple-modality features significantly outperformed all the other feature configurations. This deep learning model mimics the workflow of a human expert and leverages multimodal data from patients with PTC, thus further supporting clinical decision-making.

A Deep Multi-Task Network to Learn Tumor Pathological Representations for Lymph Node Metastasis Prediction.

3D multi-scale, multi-task, and multi-label deep learning for prediction of lymph node metastasis in T1 lung adenocarcinoma patients' CT images

A cross-modal 3D deep learning for accurate lymph node metastasis prediction in clinical stage T1 lung adenocarcinoma

Multi-task Adaptive Resolution Network for Lymph Node Metastasis Diagnosis from Whole Slide Images of Colorectal Cancer

Lymph Node Gross Tumor Volume Detection in Oncology Imaging via Relationship Learning Using Graph Neural Network

A Multi-Task Convolutional Neural Network for Lesion Region Segmentation and Classification of Non-Small Cell Lung Carcinoma

Unsupervised domain selective graph convolutional network for preoperative prediction of lymph node metastasis in gastric cancer

Preoperative Prediction of Lymph Node Metastasis in Colorectal Cancer with Deep Learning

A deep local attention network for pre-operative lymph node metastasis prediction in pancreatic cancer via multiphase CT imaging

Deep learning approach for predicting lymph node metastasis in non-small cell lung cancer by fusing image–gene data

Integrating features from lymph node stations for metastatic lymph node detection

Prediction of lymph node metastasis in primary gastric cancer from pathological images and clinical data by multimodal multiscale deep learning

A Multi-Modal Heterogeneous Graph Forest to Predict Lymph Node Metastasis of Non-Small Cell Lung Cancer

Predicting Gastric Cancer Outcome from Resected Lymph Node Histopathology Images Using Deep Learning.

A deep learning method for predicting the origins of cervical lymph node metastatic cancer on digital pathological images

Development of a multi-modal learning-based lymph node metastasis prediction model for lung cancer

Deep learning model based on primary tumor to predict lymph node status in clinical stage IA lung adenocarcinoma: a multicenter study

Using Natural Language Processing and Machine Learning to Preoperatively Predict Lymph Node Metastasis for Non–Small Cell Lung Cancer With Electronic Medical Records: Development and Validation Study

A model for predicting lymph node metastasis of thyroid carcinoma: a multimodality convolutional neural network study

Identification of lymph node metastasis in pre‐operation cervical cancer patients by weakly supervised deep learning from histopathological whole‐slide biopsy images

A Novel Deep Learning Framework Based Mask-Guided Attention Mechanism for Distant Metastasis Prediction of Lung Cancer