Abstract:Background: Patients with malignant tumors often develop bone metastases. SPECT bone scintigraphy is an effective tool for surveying bone metastases due to its high sensitivity, low-cost equipment, and radiopharmaceutical. However, the low spatial resolution of SPECT scans significantly hinders manual analysis by nuclear medicine physicians. Deep learning, a promising technique for automated image analysis, can extract hierarchal patterns from images without human intervention. Objective: To enhance the performance of deep learning-based segmentation models, we integrate textual data from diagnostic reports with SPECT bone scans, aiming to develop an automated analysis method that outperforms purely unimodal data-driven segmentation models. Methods: We propose a dual-path segmentation framework to extract features from bone scan images and diagnostic reports separately. In the first path, an encoder-decoder network is employed to learn hierarchical representations of features from SPECT bone scan images. In the second path, the Chinese version of the MacBERT model is utilized to develop a text encoder for extracting features from diagnostic reports. The extracted textual features are then fused with image features during the decoding stage in the first path, enhancing the overall segmentation performance. Results: Experimental evaluation conducted on real-world clinical data demonstrated the superior performance of the proposed segmentation model. Our model achieved a 0.0209 increase in the DSC (Dice Similarity Coefficient) score compared to the well-known U-Net model. Conclusions: The proposed multimodal data-driven method effectively identifies and isolates metastasis lesions in SPECT bone scans, outperforming existing classical deep learning models. This study demonstrates the value of incorporating textual data in the deep learning-based segmentation of lowresolution SPECT bone scans.

[Hearing status after stapedectomy with preservation or cutting of stapedius tendon].

Improved UNet‐based Magnetic Resonance Imaging Segmentation of Demyelinating Diseases with Small Lesion Regions

BMSMM-Net: A Bone Metastasis Segmentation Framework Based on Mamba and Multiperspective Extraction

Mmy-net: a Multimodal Network Exploiting Image and Patient Metadata for Simultaneous Segmentation and Diagnosis

MMFNet: A multi-modality MRI fusion network for segmentation of nasopharyngeal carcinoma

Combination and evolution of HBV mutant strains in the HBeAg-positive status predict clinical outcomes after HBeAg seroconversion

AugMS-Net:Augmented Multiscale Network for Small Cervical Tumor Segmentation from MRI Volumes

AAMR-FCN myeloma cancer net: Adaptive and attention-based mask R-FCN for diagnosing myeloma cancer using cell microscopic images with hybrid heuristic strategy

MM-UNet: A multimodality brain tumor segmentation network in MRI images

A dual-branch hybrid dilated CNN model for the AI-assisted segmentation of meningiomas in MR images

Splenomegaly Segmentation on Multi-modal MRI using Deep Convolutional Networks

Activation of Tolloid‐like 1 gene expression by the cardiac specific homeobox gene Nkx2–5

Multi-scale Regional Attention Deeplab3+: Multiple Myeloma Plasma Cells Segmentation in Microscopic Images

Attention-enhanced multiscale feature fusion network for pancreas and tumor segmentation

Aided Diagnosis Model Based on Deep Learning for Glioblastoma, Solitary Brain Metastases, and Primary Central Nervous System Lymphoma with Multi-Modal MRI

Automatic cervical cancer segmentation in multimodal magnetic resonance imaging using an EfficientNet encoder in UNet++ architecture

Multimodal Data-Driven Segmentation of Bone Metastasis Lesions in SPECT Bone Scans Using Deep Learning

Automated Whole-Body Bone Lesion Detection for Multiple Myeloma on 68Ga-Pentixafor PET/CT Imaging Using Deep Learning Methods.

Calcareous degeneration of the canine cornea.

Context Multi-scale Fusion Network for Magnetic Resonance Image

Automatic segmentation of rectal tumors from MRI using multiscale densely connected convolutional neural network based on attention mechanism