Abstract:In this paper, we propose a novel image coding framework with semantic-aware visual decomposition towards extremely low bitrate compression. In particular, an input image is analyzed into a semantic map as structural representation and semantic-wise texture representation and further compressed into bitstreams at the encoder side. On the decoder side, the received bitstreams of dual-layer representations are decoded and reconstructed for target image synthesis with generative models. Moreover, the attention mechanism is introduced into the model architecture for texture representation modeling and a coherency regularization is proposed to further optimize the texture representation space by aligning the representation space with the source pixel space for higher synthesis quality. Besides, we also propose a cross-channel entropy module and control the quantization scale to facilitate rate-distortion optimization. Upon compressing the decomposed components into the bitstream, the simple yet effective representation philosophy benefits image compression in many aspects. First, in terms of compression performance, compact representations, and high visual synthesis quality can bring remarkable advantages. Second, the proposed framework yields a physically explainable bitstream composed of the structural segment and semantic-wise texture segments. Third and most importantly, subsequent vision tasks (e.g., content manipulation) can receive fundamental support from the semantic-aware visual decomposition and synthesis mechanism. Extensive experimental results demonstrate the superiority of the proposed framework towards efficient visual representation learning, high efficiency image compression ( bpp), and intelligent visual applications (e.g., manipulation and analysis).

Semantic Compression with Information Lattice Learning

Information Compression in the AI Era: Recent Advances and Future Challenges

Semantic Arithmetic Coding using Synonymous Mappings

Information-Theoretic Limits on Compression of Semantic Information

Compression Represents Intelligence Linearly

Semantic-assisted image compression

Approaching Rate-Distortion Limits in Neural Compression with Lattice Transform Coding

DeepSIC: Deep Semantic Image Compression

Towards Semantic Communications: Deep Learning-Based Image Semantic Coding

Semantic Ensemble Loss and Latent Refinement for High-Fidelity Neural Image Compression

Understanding is Compression

Toward Semantic Communications: Deep Learning-Based Image Semantic Coding

Unified Architecture Adaptation for Compressed Domain Semantic Inference

Information-Ordered Bottlenecks for Adaptive Semantic Compression

Learning Optimal Lattice Vector Quantizers for End-to-end Neural Image Compression

Compression Ratio Learning and Semantic Communications for Video Imaging

Semantic Compression with Side Information: A Rate-Distortion Perspective

Learning based Facial Image Compression with semantic fidelity metric

LCS: Learning Compressible Subspaces for Adaptive Network Compression at Inference Time

Semantic-Aware Visual Decomposition for Image Coding