Abstract:In breast cancer research, diverse data types and formats, such as radiological images, clinical records, histological data, and expression analysis, are employed. Given the intricate nature of natural phenomena, relying on the features of a single modality is seldom sufficient for comprehensive analysis. Therefore, it is possible to guarantee medical relevance and achieve improved clinical outcomes by combining several modalities. The presen study carefully maps and reviews 47 primary articles from six well-known digital libraries that were published between 2018 and 2023 for breast cancer classification based on multimodal deep learning fusion (MDLF) techniques. This systematic literature review encompasses various aspects, including the medical modalities combined, the datasets utilized in these studies, the techniques, models, and architectures used in MDLF and it also discusses the advantages and limitations of each approach. The analysis of selected papers has revealed a compelling trend: the emergence of new modalities and combinations that were previously unexplored in the context of breast cancer classification. This exploration has not only expanded the scope of predictive models but also introduced fresh perspectives for addressing diverse targets, ranging from screening to diagnosis and prognosis. The practical advantages of MDLF are evident in its ability to enhance the predictive capabilities of machine learning models, resulting in improved accuracy across diverse applications. The prevalence of deep learning models underscores their success in autonomously discerning complex patterns, offering a substantial departure from traditional machine learning approaches. Furthermore, the paper explores the challenges and future directions in this field, including the need for larger datasets, the use of ensemble learning methods, and the interpretation of multimodal models.

Multi-modal Medical Data Fusion using Deep Learning

Multimodal medical image fusion and classification using deep learning techniques

Multimodal Medical Imaging Using Modern Deep Learning Approaches

Multimodal Medical Image Fusion: The Perspective of Deep Learning

Multimodal Fusion Learning with Dual Attention for Medical Imaging

Adaptive Multimodal Image Fusion with a Deep Pyramidal Residual Learning Network

Multimodal medical image fusion algorithm in the era of big data

Deep Multi-modal Fusion of Image and Non-image Data in Disease Diagnosis and Prognosis: A Review

Integrating Medical Imaging and Clinical Reports Using Multimodal Deep Learning for Advanced Disease Analysis

Multimodal fusion with deep neural networks for leveraging CT imaging and electronic health record: a case-study in pulmonary embolism detection

An efficient Dense-Resnet for multimodal image fusion using medical image

A comprehensive investigation of multimodal deep learning fusion strategies for breast cancer classification

Missing-modality Enabled Multi-modal Fusion Architecture for Medical Data

Multi-modal Medical Neurological Image Fusion using Wavelet Pooled Edge Preserving Autoencoder

Application of Multimodal Fusion Deep Learning Model in Disease Recognition

Feature extraction of multimodal medical image fusion using novel deep learning and contrast enhancement method

Deep learning‐based multimodal fusion network for segmentation and classification of breast cancers using B‐mode and elastography ultrasound images

Multimodal medical image fusion using convolutional neural network and extreme learning machine

Transplant-related Kaposi sarcoma in children.

A review of deep learning-based information fusion techniques for multimodal medical image classification

An overview of deep learning methods for multimodal medical data mining