Abstract:The emerging field semantic communication is driving the research of end-to-end data transmission. By utilizing the powerful representation ability of deep learning models, learned data transmission schemes have exhibited superior performance than the established source and channel coding methods. While, so far, research efforts mainly concentrated on architecture and model improvements toward a static target domain. Despite their successes, such learned models are still suboptimal due to the limitations in model capacity and imperfect optimization and generalization, particularly when the testing data distribution or channel response is different from that adopted for model training, as is likely to be the case in real-world. To tackle this, in this paper, we propose a novel online learned joint source and channel coding approach that leverages the deep learning model's overfitting property. Specifically, we update the off-the-shelf pre-trained models after deployment in a lightweight online fashion to adapt to the distribution shifts in source data and environment domain. We take the overfitting concept to the extreme, proposing a series of implementation-friendly methods to adapt the codec model or representations to an individual data or channel state instance, which can further lead to substantial gains in terms of the end-to-end rate-distortion performance. Accordingly, the streaming ingredients include both the semantic representations of source data and the online updated decoder model parameters. The system design is formulated as a joint optimization problem whose goal is to minimize the loss function, a tripartite trade-off among the data stream bandwidth cost, model stream bandwidth cost, and end-to-end distortion. The proposed methods enable the communication-efficient adaptation for all parameters in the network without sacrificing decoding speed. Extensive experiments, including user study, on continually changing target source data and wireless channel environments, demonstrate the effectiveness and efficiency of our approach, on which we outperform existing state-of-the-art engineered transmission scheme (VVC combined with 5G LDPC coded transmission).

Learned Image Transmission over MIMO Fading Channels

Versatile Semantic Coded Transmission over MIMO Fading Channels

Pilot-Free Semantic Communication over Multi-User Mimo Fading Channels

Learned Image Transmission Toward Machine-Type Semantic Communications

A Deep Learning Based Broadcast Approach for Image Semantic Communication over Fading Channels

Benchmarking Semantic Communications for Image Transmission Over MIMO Interference Channels

Deep Joint Source-Channel Coding for Adaptive Image Transmission over MIMO Channels

Robust Image Semantic Coding with Learnable CSI Fusion Masking over MIMO Fading Channels

Progressive Image Transmission over Space-Time Coded OFDM-based MIMO Systems with Adaptive Modulation

Simultaneous Federated Learning and Information Transmission over Time-Varying MIMO Channels.

MIMO Visible Light Communication System with Block Markov Superposition Transmission

Wireless Transmission of Images With The Assistance of Multi-level Semantic Information

Deep Joint Semantic Coding and Beamforming for Near-Space Airship-Borne Massive MIMO Network

A Video Transmission Scheme Based on Scalable Video Coding over MIMO Wireless Systems

Robust Image Transmission over Mimo Space-Time Coded Wireless Systems*

Data Adaptive Semantic Communication Systems for Intelligent Tasks and Image Transmission

Toward Adaptive Semantic Communications: Efficient Data Transmission Via Online Learned Nonlinear Transform Source-Channel Coding

Fusion-Based Multi-User Semantic Communications for Wireless Image Transmission over Degraded Broadcast Channels

Goal-Oriented Semantic Information Transmission With Message-Sharing NOMA

DM-MIMO: Diffusion Models for Robust Semantic Communications over MIMO Channels

Optimization of Image Transmission in Cooperative Semantic Communication Networks