Abstract:Semantic communication (SemCom) has emerged as a promising technique for the next-generation communication systems, in which the generation at the receiver side is allowed without semantic features' recovery. However, the majority of existing research predominantly utilizes a singular type of semantic information, such as text, images, or speech, to supervise and choose the generated source signals, which may not sufficiently encapsulate the comprehensive and accurate semantic information, and thus creating a performance bottleneck. In order to bridge this gap, in this paper, we propose and investigate a multimodal information-aided SemCom framework (MMSemCom) for image transmission. To be specific, in this framework, we first extract semantic features at both the image and text levels utilizing the Convolutional Neural Network (CNN) architecture and the Contrastive Language-Image Pre-Training (CLIP) model before transmission. Then, we employ a generative diffusion model at the receiver to generate multiple images. In order to ensure the accurate extraction and facilitate high-fidelity image reconstruction, we select the "best" image with the minimum reconstruction errors by taking both the aided image and text semantic features into account. We further extend MMSemCom to the multiuser scenario for orthogonal transmission. Experimental results demonstrate that the proposed framework can not only achieve the enhanced fidelity and robustness in image transmission compared with existing communication systems but also sustain a high performance in the low signal-to-noise ratio (SNR) conditions.

Semantic Communications for Multi-View Generation

Multi-User Probabilistic Semantic Communication with Semantic Compression Ratio Optimization

Semantic Feature Scheduling and Rate Control in Multi-Modal Distributed Network

Green Probabilistic Semantic Communication over Wireless Networks

A Joint Communication and Computation Design for Probabilistic Semantic Communications

Spectral Efficiency Maximization for Probabilistic Semantic Communication with Rate Splitting

Joint Compression Ratio and User Association for Multi-Cell Probabilistic Semantic Communication

Knowledge-Aided Semantic Communication Leveraging Probabilistic Graphical Modeling

Generative Semantic Communications with Foundation Models: Perception-Error Analysis and Semantic-Aware Power Allocation

Sum Rate Maximization for Distributed Riss Assisted Probabilistic Semantic Communication

Semantic Communication for Cooperative Perception based on Importance Map

Semantic Successive Refinement: A Generative AI-aided Semantic Communication Framework

From Analog to Digital: Multi-Order Digital Joint Coding-Modulation for Semantic Communication

Probabilistic Modeling of Semantic Ambiguity for Scene Graph Generation

Versatile Semantic Coded Transmission over MIMO Fading Channels

Multi-User Semantic Fusion for Semantic Communications over Degraded Broadcast Channels

Position-Aided Semantic Communication for Efficient Image Transmission: Design, Implementation, and Experimental Results

Image Generation with Multimodule Semantic Feature-Aided Selection for Semantic Communications

Fusion-Based Multi-User Semantic Communications for Wireless Image Transmission over Degraded Broadcast Channels

Cross-Modal Semantic Communications