Abstract:Background and Objective Thyroid nodule segmentation is a crucial step in the diagnostic procedure of physicians and computer-aided diagnosis systems. However, prevailing studies often treat segmentation and diagnosis as independent tasks, overlooking the intrinsic relationship between these processes. The sequence steps of these independent tasks in computer-aided diagnosis systems may lead to the accumulation of errors. Therefore, it is worth combining them as a whole by exploring the relationship between thyroid nodule segmentation and diagnosis. According to the diagnostic procedure of thyroid imaging reporting and data system (TI-RADS), the assessment of shape and margin characteristics is the prerequisite for radiologists to discriminate benign and malignant thyroid nodules. Inspired by TI-RADS, this study aims to combine these tasks into a whole through the prior knowledge from TI-RADS, thereby enhancing the accuracy and interpretability of thyroid nodule analysis. Methods Specifically, this paper proposes a shape-margin knowledge augmented network (SkaNet) for simultaneous thyroid nodule segmentation and diagnosis. Due to the visual feature similarities between segmentation and diagnosis, SkaNet shares visual features in the feature extraction stage and then utilizes a dual-branch architecture to perform thyroid nodule segmentation and diagnosis tasks respectively. In the shared feature extraction, the combination of convolutional feature maps and self-attention maps allows to exploitation of both local information and global patterns in thyroid nodule images. To enhance effective discriminative features, an exponential mixture module is introduced, combining convolutional feature maps and self-attention maps through exponential weighting. Then, SkaNet is jointly optimized by a knowledge augmented multi-task loss function with a constraint penalty term. The constraint penalty term embeds shape and margin characteristics through numerical computations, establishing a vital relationship between thyroid nodule diagnosis results and segmentation masks. Results We evaluate the proposed approach on a public thyroid ultrasound dataset (DDTI) and a locally collected thyroid ultrasound dataset. The experimental results reveal the value of our contributions and demonstrate that our approach can yield significant improvements compared with state-of-the-art counterparts. Conclusions SkaNet highlights the potential of combining thyroid nodule segmentation and diagnosis with knowledge augmented learning into a unified framework, which captures the shape and margin characteristics critical for discriminating benign and malignant thyroid nodules. Our findings suggest promising insights for advancing computer-aided diagnosis joint with segmentation.

Morphology-Aware Interactive Keypoint Estimation

Deep neural models for automated multi-task diagnostic scan management—quality enhancement, view classification and report generation

Neural Interactive Keypoint Detection

Bones Can't Be Triangles: Accurate and Efficient Vertebrae Keypoint Estimation through Collaborative Error Revision

Automatic quantification of morphology on magnetic resonance images of the proximal tibia

Automated Diagnosis of Bone Metastasis Based on Multi-View Bone Scans Using Attention-Augmented Deep Neural Networks

Bidirectional meta-Kronecker factored optimizer and Hausdorff distance loss for few-shot medical image segmentation

Iterative annotation to ease neural network training: Specialized machine learning in medical image analysis

Correction of Two Human Key-Points Estimations for Medical Applications

A Small Target Segmentation-based Assistive Medical Diagnosis Method via Multimedia Data Analysis

Efficient and Generic Interactive Segmentation Framework to Correct Mispredictions during Clinical Evaluation of Medical Images

Progressive Medical Image Annotation with Convolutional Neural Network-Based Interactive Segmentation Method

Fast interactive medical image segmentation with weakly supervised deep learning method

An Efficient Neural Network for Segmenting Multiple Joint Tissues From Knee MRI With Hyperparameter Optimization: Data From the Osteoarthritis Initiative

Interactive segmentation of medical images through fully convolutional neural networks

Shape-Margin Knowledge Augmented Network for Thyroid Nodule Segmentation and Diagnosis

A Robust and Interpretable Deep Learning Framework for Multi-modal Registration via Keypoints

K-Diag: Knowledge-enhanced Disease Diagnosis in Radiographic Imaging

SEKD: Self-Evolving Keypoint Detection and Description

KCB-Net: A 3D knee cartilage and bone segmentation network via sparse annotation

Multilevel Attention Siamese Network for Keypoint Detection in Optical and SAR Images