Abstract:In this paper, we propose a novel image coding framework with semantic-aware visual decomposition towards extremely low bitrate compression. In particular, an input image is analyzed into a semantic map as structural representation and semantic-wise texture representation and further compressed into bitstreams at the encoder side. On the decoder side, the received bitstreams of dual-layer representations are decoded and reconstructed for target image synthesis with generative models. Moreover, the attention mechanism is introduced into the model architecture for texture representation modeling and a coherency regularization is proposed to further optimize the texture representation space by aligning the representation space with the source pixel space for higher synthesis quality. Besides, we also propose a cross-channel entropy module and control the quantization scale to facilitate rate-distortion optimization. Upon compressing the decomposed components into the bitstream, the simple yet effective representation philosophy benefits image compression in many aspects. First, in terms of compression performance, compact representations, and high visual synthesis quality can bring remarkable advantages. Second, the proposed framework yields a physically explainable bitstream composed of the structural segment and semantic-wise texture segments. Third and most importantly, subsequent vision tasks (e.g., content manipulation) can receive fundamental support from the semantic-aware visual decomposition and synthesis mechanism. Extensive experimental results demonstrate the superiority of the proposed framework towards efficient visual representation learning, high efficiency image compression ( bpp), and intelligent visual applications (e.g., manipulation and analysis).

Image Coding and Compression Method Based on Semantics of Pixels for Semantic Communication

Semantic Communication Approach for Multi-Task Image Transmission

Deep Joint Source-Channel Coding Based on Semantics of Pixels

Semantic-assisted image compression

Towards Semantic Communications: Deep Learning-Based Image Semantic Coding

Toward Semantic Communications: Deep Learning-Based Image Semantic Coding

Deep Learning-Based Image Semantic Coding for Semantic Communications

Towards Semantically Scalable Image Coding Using Semantic Map.

Deep Learning-Based Image Semantic Communication System

A Knowledge Base Driven Task-Oriented Image Semantic Communication Scheme

Semantics-to-Signal Scalable Image Compression with Learned Revertible Representations

Semantic Scalable Image Compression with Cross-Layer Priors.

DeepSIC: Deep Semantic Image Compression

Deep Joint Source-Channel Coding for Wireless Image Transmission with Semantic Importance

Semantic-Aware Visual Decomposition for Image Coding

Rethinking Semantic Image Compression: Scalable Representation with Cross-modality Transfer

An End-to-End Deep Learning Image Compression Framework Based on Semantic Analysis

Task-Oriented Scene Graph-Based Semantic Communications with Adaptive Channel Coding

A Robust Semantic Communication System for Image

Task-Oriented Explainable Semantic Communications Based on Structured Scene Graphs

SSSIC: Semantics-to-Signal Scalable Image Coding with Learned Structural Representations.