Deep Learning Based Multimodal Biomedical Data Fusion: an Overview and Comparative Review

Junwei Duan,Jiaqi Xiong,Yinghui Li,Weiping Ding

DOI: https://doi.org/10.1016/j.inffus.2024.102536

IF: 18.6

2024-01-01

Information Fusion

Abstract:Multimodal biomedical data fusion plays a pivotal role in distilling comprehensible and actionable insights by seamlessly integrating disparate biomedical data from multiple modalities, effectively circumventing the constraints of single-modal approaches. Over recent decades, the proliferation of biomedical data availability and the advent of advanced techniques such as deep learning and large-scale models have underscored the critical importance of multimodal biomedical data fusion across diverse applications. However, despite its burgeoning significance, several theoretical challenges persist within this domain. To forge a more robust theoretical framework for multimodal biomedical data fusion, this survey endeavors to conduct a comprehensive investigation, with a specific focus on diverse biomedical data representations and fusion methodologies anchored in deep learning. Commencing with a succinct exposition of commonly employed biomedical data representations, we also curate and synthesize a selection of prominent and invaluable datasets that serve as foundational cornerstones for multimodal biomedical data fusion endeavors. Subsequently, we embark on a meticulous examination of extant classical deep learning-based methods and multimodal biomedical data fusion techniques, meticulously categorizing them according to three distinct fusion levels, respectively. Lastly, we illuminate potential challenges stemming from the advent of cutting-edge technologies such as generative models and large language models, elucidating their implications for multimodal biomedical data fusion.