Abstract:Contrast-enhanced ultrasound (CEUS) has been extensively employed as an imaging modality in thyroid nodule diagnosis due to its capacity to visualise the distribution and circulation of micro-vessels in organs and lesions in a non-invasive manner. However, current CEUS-based thyroid nodule diagnosis methods suffered from: 1) the blurred spatial boundaries between nodules and other anatomies in CEUS videos, and 2) the insufficient representations of the local structural information of nodule tissues by the features extracted only from CEUS videos. In this paper, we propose a novel dual-branch network with a cross-modality-attention mechanism for thyroid nodule diagnosis by integrating the information from tow related modalities, i.e., CEUS videos and ultrasound image. The mechanism has two parts: US-attention-from-CEUS transformer (UAC-T) and CEUS-attention-from-US transformer (CAU-T). As such, this network imitates the manner of human radiologists by decomposing the diagnosis into two correlated tasks: 1) the spatio-temporal features extracted from CEUS are hierarchically embedded into the spatial features extracted from US with UAC-T for the nodule segmentation; 2) the US spatial features are used to guide the extraction of the CEUS spatio-temporal features with CAU-T for the nodule classification. The two tasks are intertwined in the dual-branch end-to-end network and optimized with the multi-task learning (MTL) strategy. The proposed method is evaluated on our collected thyroid US-CEUS dataset. Experimental results show that our method achieves the classification accuracy of 86.92%, specificity of 66.41%, and sensitivity of 97.01%, outperforming the state-of-the-art methods. As a general contribution in the field of multi-modality diagnosis of diseases, the proposed method has provided an effective way to combine static information with its related dynamic information, improving the quality of deep learning based diagnosis with an additional benefit of explainability.

CEUSegNet: A Cross-Modality Lesion Segmentation Network for Contrast-Enhanced Ultrasound

CEUS-Net: Lesion Segmentation in Dynamic Contrast-Enhanced Ultrasound with Feature-Reweighted Attention Mechanism.

Attention-Guided Network with Densely Connected Convolution for Skin Lesion Segmentation

GSCEU-Net: an End-to-End Lightweight Skin Lesion Segmentation Model with Feature Fusion Based on U-Net Enhancements

ECU-Net: multi-scale salient boundary detection and contrast feature enhancement U-Net for breast ultrasound image segmentation

Dynamic Perfusion Representation and Aggregation Network for Nodule Segmentation Using Contrast-Enhanced US.

A Hybrid Enhanced Attention Transformer Network for Medical Ultrasound Image Segmentation

CSS-Net: A Collaborative Framework for Synthesis and Segmentation of Missing Contrast-enhanced Image with Error-Prediction Consistency

Effective hybrid attention network based on pseudo-color enhancement in ultrasound image segmentation

CUSS-Net: A Cascaded Unsupervised-Based Strategy and Supervised Network for Biomedical Image Diagnosis and Segmentation.

A Cross-attention and Multi-level Feature Fusion Network for Breast Lesion Segmentation in Ultrasound Images

A Multiscale Nonlocal Feature Extraction Network for Breast Lesion Segmentation in Ultrasound Images.

MLFEU-NET: A Multi-scale Low-level Feature Enhancement Unet for breast lesions segmentation in ultrasound images

GBE-Net: Global Boundary Enhancement Network for breast lesion segmentation in ultrasound images

FECC-Net: A Novel Feature Enhancement and Context Capture Network Based on Brain MRI Images for Lesion Segmentation

A Dual-Branch Cross-Modality-Attention Network for Thyroid Nodule Diagnosis Based on Ultrasound Images and Contrast-Enhanced Ultrasound Videos

CE-Net: Context Encoder Network for 2D Medical Image Segmentation

CDSE-UNet: Enhancing COVID-19 CT Image Segmentation with Canny Edge Detection and Dual-Path SENet Feature Fusion

Lesion Segmentation in Ultrasound Using Semi-pixel-wise Cycle Generative Adversarial Nets

Ssm-Net: Semi-Supervised Multi-Task Network for Joint Lesion Segmentation and Classification from Pancreatic Eus Images