Abstract:In breast cancer research, diverse data types and formats, such as radiological images, clinical records, histological data, and expression analysis, are employed. Given the intricate nature of natural phenomena, relying on the features of a single modality is seldom sufficient for comprehensive analysis. Therefore, it is possible to guarantee medical relevance and achieve improved clinical outcomes by combining several modalities. The presen study carefully maps and reviews 47 primary articles from six well-known digital libraries that were published between 2018 and 2023 for breast cancer classification based on multimodal deep learning fusion (MDLF) techniques. This systematic literature review encompasses various aspects, including the medical modalities combined, the datasets utilized in these studies, the techniques, models, and architectures used in MDLF and it also discusses the advantages and limitations of each approach. The analysis of selected papers has revealed a compelling trend: the emergence of new modalities and combinations that were previously unexplored in the context of breast cancer classification. This exploration has not only expanded the scope of predictive models but also introduced fresh perspectives for addressing diverse targets, ranging from screening to diagnosis and prognosis. The practical advantages of MDLF are evident in its ability to enhance the predictive capabilities of machine learning models, resulting in improved accuracy across diverse applications. The prevalence of deep learning models underscores their success in autonomously discerning complex patterns, offering a substantial departure from traditional machine learning approaches. Furthermore, the paper explores the challenges and future directions in this field, including the need for larger datasets, the use of ensemble learning methods, and the interpretation of multimodal models.

Deep Multi-modal Fusion of Image and Non-image Data in Disease Diagnosis and Prognosis: A Review

A Comprehensive Review on Synergy of Multi-Modal Data and AI Technologies in Medical Diagnosis

Multimodal Machine Learning in Precision Health

Recent Advances in Data-driven Fusion of Multi-modal Imaging and Genomics for Precision Medicine

Advancing healthcare through multimodal data fusion: a comprehensive review of techniques and applications

Integrating Medical Imaging and Clinical Reports Using Multimodal Deep Learning for Advanced Disease Analysis

A review of deep learning-based information fusion techniques for multimodal medical image classification

Application of Multimodal Fusion Deep Learning Model in Disease Recognition

Fusion of Multi-Modality Medical Imaging Information for Clinical Decision in Tumor Diagnosis and Treatment

Multimodal medical image fusion algorithm in the era of big data

Interpretation on Deep Multimodal Fusion for Diagnostic Classification

Deep Learning Based Multimodal Biomedical Data Fusion: an Overview and Comparative Review

Enhancing Heart Failure Diagnosis Through Multi-Modal Data Integration and Deep Learning

A Review of the Application of Multi-modal Deep Learning in Medicine: Bibliometrics and Future Directions

Medical Multimodal Foundation Models in Clinical Diagnosis and Treatment: Applications, Challenges, and Future Directions

A review: Deep learning for medical image segmentation using multi-modality fusion

Histopathology in focus: a review on explainable multi-modal approaches for breast cancer diagnosis

A Review of Multimodal Medical Image Fusion Techniques

A review of cancer data fusion methods based on deep learning

Review of multimodal machine learning approaches in healthcare

A comprehensive investigation of multimodal deep learning fusion strategies for breast cancer classification