Abstract:PURPOSE: Recent studies have illustrated that the peritumoral regions of medical images have value for clinical diagnosis. However, the existing approaches using peritumoral regions mainly focus on the diagnostic capability of the single region and ignore the advantages of effectively fusing the intratumoral and peritumoral regions. In addition, these methods need accurate segmentation masks in the testing stage, which are tedious and inconvenient in clinical applications. To address these issues, we construct a deep convolutional neural network that can adaptively fuse the information of multiple tumoral-regions (FMRNet) for breast tumor classification using ultrasound (US) images without segmentation masks in the testing stage.METHODS: To sufficiently excavate the potential relationship, we design a fused network and two independent modules to extract and fuse features of multiple regions simultaneously. First, we introduce two enhanced combined-tumoral (EC) region modules, aiming to enhance the combined-tumoral features gradually. Then, we further design a three-branch module for extracting and fusing the features of intratumoral, peritumoral, and combined-tumoral regions, denoted as the intratumoral, peritumoral, and combined-tumoral module. Especially, we design a novel fusion module by introducing a channel attention module to adaptively fuse the features of three regions. The model is evaluated on two public datasets including UDIAT and BUSI with breast tumor ultrasound images. Two independent groups of experiments are performed on two respective datasets using the fivefold stratified cross-validation strategy. Finally, we conduct ablation experiments on two datasets, in which BUSI is used as the training set and UDIAT is used as the testing set.RESULTS: We conduct detailed ablation experiments about the proposed two modules and comparative experiments with other existing representative methods. The experimental results show that the proposed method yields state-of-the-art performance on both two datasets. Especially, in the UDIAT dataset, the proposed FMRNet achieves a high accuracy of 0.945 and a specificity of 0.945, respectively. Moreover, the precision (PRE = 0.909) even dramatically improves by 21.6% on the BUSI dataset compared with the existing method of the best result.CONCLUSION: The proposed FMRNet shows good performance in breast tumor classification with US images, and proves its capability of exploiting and fusing the information of multiple tumoral-regions. Furthermore, the FMRNet has potential value in classifying other types of cancers using multiple tumoral-regions of other kinds of medical images.

MFU-Net: a deep multimodal fusion network for breast cancer segmentation with dual-layer spectral detector CT

FMRNet: A fused network of multiple tumoral regions for breast tumor classification with ultrasound images

Deep learning‐based multimodal fusion network for segmentation and classification of breast cancers using B‐mode and elastography ultrasound images

MDF-Net: A Multi-Scale Dynamic Fusion Network for Breast Tumor Segmentation of Ultrasound Images

CMAF-Net: a cross-modal attention fusion-based deep neural network for incomplete multi-modal brain tumor segmentation

MDCC-Net: Multiscale double-channel convolution U-Net framework for colorectal tumor segmentation

MFMSNet: A Multi-frequency and Multi-scale Interactive CNN-Transformer Hybrid Network for breast ultrasound image segmentation

Modality-level cross-connection and attentional feature fusion based deep neural network for multi-modal brain tumor segmentation

A Cross-attention and Multi-level Feature Fusion Network for Breast Lesion Segmentation in Ultrasound Images

MFH‐Net: A Hybrid CNN‐Transformer Network Based Multi‐Scale Fusion for Medical Image Segmentation

A Multiscale Nonlocal Feature Extraction Network for Breast Lesion Segmentation in Ultrasound Images.

MM-UNet: A multimodality brain tumor segmentation network in MRI images

MCNMF-Unet: a mixture Conv-MLP network with multi-scale features fusion Unet for medical image segmentation

MFCNet: A multi-modal fusion and calibration networks for 3D pancreas tumor segmentation on PET-CT images

Modality preserving U-Net for segmentation of multimodal medical images

A novel MCF-Net: Multi-level context fusion network for 2D medical image segmentation

MF-Net: Multiple-feature Extraction Network for Breast Lesion Segmentation in Ultrasound Images

Brain tumor segmentation by combining MultiEncoder UNet with wavelet fusion

MLFEU-NET: A Multi-scale Low-level Feature Enhancement Unet for breast lesions segmentation in ultrasound images

CMU-Net: A Strong ConvMixer-based Medical Ultrasound Image Segmentation Network

Multi-Layer Feature Fusion with Cross-Channel Attention-Based U-Net for Kidney Tumor Segmentation