Abstract:PURPOSE: Recent studies have illustrated that the peritumoral regions of medical images have value for clinical diagnosis. However, the existing approaches using peritumoral regions mainly focus on the diagnostic capability of the single region and ignore the advantages of effectively fusing the intratumoral and peritumoral regions. In addition, these methods need accurate segmentation masks in the testing stage, which are tedious and inconvenient in clinical applications. To address these issues, we construct a deep convolutional neural network that can adaptively fuse the information of multiple tumoral-regions (FMRNet) for breast tumor classification using ultrasound (US) images without segmentation masks in the testing stage.METHODS: To sufficiently excavate the potential relationship, we design a fused network and two independent modules to extract and fuse features of multiple regions simultaneously. First, we introduce two enhanced combined-tumoral (EC) region modules, aiming to enhance the combined-tumoral features gradually. Then, we further design a three-branch module for extracting and fusing the features of intratumoral, peritumoral, and combined-tumoral regions, denoted as the intratumoral, peritumoral, and combined-tumoral module. Especially, we design a novel fusion module by introducing a channel attention module to adaptively fuse the features of three regions. The model is evaluated on two public datasets including UDIAT and BUSI with breast tumor ultrasound images. Two independent groups of experiments are performed on two respective datasets using the fivefold stratified cross-validation strategy. Finally, we conduct ablation experiments on two datasets, in which BUSI is used as the training set and UDIAT is used as the testing set.RESULTS: We conduct detailed ablation experiments about the proposed two modules and comparative experiments with other existing representative methods. The experimental results show that the proposed method yields state-of-the-art performance on both two datasets. Especially, in the UDIAT dataset, the proposed FMRNet achieves a high accuracy of 0.945 and a specificity of 0.945, respectively. Moreover, the precision (PRE = 0.909) even dramatically improves by 21.6% on the BUSI dataset compared with the existing method of the best result.CONCLUSION: The proposed FMRNet shows good performance in breast tumor classification with US images, and proves its capability of exploiting and fusing the information of multiple tumoral-regions. Furthermore, the FMRNet has potential value in classifying other types of cancers using multiple tumoral-regions of other kinds of medical images.

Learning Multiparametric Biomarkers for Assessing MR-Guided Focused Ultrasound Treatment of Malignant Tumors

Deep Learning-Based Automatic Segmentation for Size and Volumetric Measurement of Breast Cancer on Magnetic Resonance Imaging.

Histology to 3D In Vivo MR Registration for Volumetric Evaluation of MRgFUS Treatment Assessment Biomarkers

Multiparametric Ultrasound Imaging of Prostate Cancer Using Deep Neural Networks

Multimodal Deep Learning to Differentiate Tumor Recurrence from Treatment Effect in Human Glioblastoma

Feasibility of Multimodal MRI-Based Deep Learning Prediction of High Amino Acid Uptake Regions and Survival in Patients With Glioblastoma

Biomarkers and focused ultrasound: the future of liquid biopsy for brain tumor patients

Automated Tumor and FUS Lesion Quantification on Multi-frequency Harmonic Motion and B-mode Imaging Using a Multi-modality Neural Network

Prior Clinico‐Radiological Features Informed Multi‐Modal MR Images Convolution Neural Network: A novel deep learning framework for prediction of lymphovascular invasion in breast cancer

Combined imaging biomarkers for therapy evaluation in glioblastoma multiforme: correlating sodium MRI and F-18 FLT PET on a voxel-wise basis

MR-guided high-intensity focused ultrasound: current status of an emerging technology

Distinguishing Tumor Cell Infiltration and Vasogenic Edema in the Peritumoral Region of Glioblastoma at the Voxel Level via Conventional MRI Sequences

FMRNet: A fused network of multiple tumoral regions for breast tumor classification with ultrasound images

Multi-modal fusion and feature enhancement U-Net coupling with stem cell niches proximity estimation for voxel-wise GBM recurrence prediction

Evaluation of MR anatomically-guided PET reconstruction using a convolutional neural network in PSMA patients

A Novel Tumor Progression Prediction Method for Multimode Ablation Treatment

MR-guided focused ultrasound liquid biopsy enriches circulating biomarkers in patients with brain tumors

Texture Analysis of Fat-Suppressed T2-Weighted Magnetic Resonance Imaging and Use of Machine Learning to Discriminate Nasal and Paranasal Sinus Small Round Malignant Cell Tumors

MR Imaging-Guided Focused Ultrasound for Breast Tumors

Predicting peritumoral glioblastoma infiltration and subsequent recurrence using deep-learning-based analysis of multi-parametric magnetic resonance imaging

Deep learning prediction of pathological complete response, residual cancer burden, and progression-free survival in breast cancer patients