Unraveling Complexity: an Exploration into the Large-Scale Multi-Modal Signal Processing
Zhenyu Wen,Yuheng Ye,Jie Su,Taotao Li,Jinhao Wan,Shilian Zheng,Zhen Hong,Shibo He,Haoran Duan,Yuexiang Li,Yawen Huang,Yefeng Zheng
DOI: https://doi.org/10.1109/mis.2024.3398592
IF: 6.744
2024-01-01
IEEE Intelligent Systems
Abstract:Advanced communication systems and military reconnaissance are increasingly prevalent in high-tech environments, greatly supported by the flourishing in signal processing technologies. The recent exponential proliferation of sensors led to an unprecedented expansion in the scale and diversity of signals across various modalities. Such influx poses significant challenges in effectively integrating multi-modal signal data to deliver comprehensive and interpretive solutions across a diverse range of applications. In this paper, we provide an overview of the core issues, challenges, and future research directions in different stages of developing large-scale multi-modal signal processing models. Additionally, we introduce a prior investigation into signal representation learning, where we propose a contrastive learning-based framework to extract fine-grained signal features under few-shot conditions. Our proposed framework achieves a 24.1% performance improvement over baseline approaches, consistently demonstrating superiority over state-of-the-art methods. The code is accessible in this repository: https://github.com/YYH211/LSM .