Low-latency MLLM Inference with Spatiotemporal Heterogeneous Distributed Multimodal Data

Xiangrui Xu,Sicong Liu,Zhiwen Yu,Lehao Wang,Bin Gu
DOI: https://doi.org/10.1109/cscaiot62585.2024.00009
2024-01-01
Abstract:Distributed sensing systems have been widely applied in various Internet of Things (IoT) scenarios, and the emergence of the Multimodal Large Language Model (MLLM) has opened up new possibilities for these systems. However, the spatiotemporal heterogeneity and asynchronous arrival of distributed mobile data make achieving low-latency, high-accuracy MLLM inference extremely challenging. In this paper, we propose a framework of MLLM inference with spatiotemporal heterogeneous distributed data to achieve low-latency, high-accuracy MLLM inference in distributed sensing systems.
What problem does this paper attempt to address?