Abstract:In this paper, we propose a novel image coding framework with semantic-aware visual decomposition towards extremely low bitrate compression. In particular, an input image is analyzed into a semantic map as structural representation and semantic-wise texture representation and further compressed into bitstreams at the encoder side. On the decoder side, the received bitstreams of dual-layer representations are decoded and reconstructed for target image synthesis with generative models. Moreover, the attention mechanism is introduced into the model architecture for texture representation modeling and a coherency regularization is proposed to further optimize the texture representation space by aligning the representation space with the source pixel space for higher synthesis quality. Besides, we also propose a cross-channel entropy module and control the quantization scale to facilitate rate-distortion optimization. Upon compressing the decomposed components into the bitstream, the simple yet effective representation philosophy benefits image compression in many aspects. First, in terms of compression performance, compact representations, and high visual synthesis quality can bring remarkable advantages. Second, the proposed framework yields a physically explainable bitstream composed of the structural segment and semantic-wise texture segments. Third and most importantly, subsequent vision tasks (e.g., content manipulation) can receive fundamental support from the semantic-aware visual decomposition and synthesis mechanism. Extensive experimental results demonstrate the superiority of the proposed framework towards efficient visual representation learning, high efficiency image compression ( bpp), and intelligent visual applications (e.g., manipulation and analysis).

Seam carving extension: a compression perspective.

Optimal Bi-Directional Seam Carving for Compressibility-Aware Image Retargeting

Content-Based Image Compression for Arbitrary-Resolution Display Devices

A New Improved Seam Carving Content Aware Image Resizing Method

Image resizing based on combination of Seam Carving and salient region analysis

Improved Image Resizing Using Seam Carving And Scaling

Seamlet Carving for Shape-Aware Image Resizing

Semantic-Aware Visual Decomposition for Image Coding

Accumulative Energy-Based Seam Carving for Image Resizing

Edge-oriented Compression Coding on Image Sequence

Depth Image Sequences Compression Using Programmable Graphics Hardware

A Kind of Spatial Scalable Image Coding Scheme Based on Content

Seam Carving for Image Resizing Based on Saliency

Improved seam carving for content-aware image retargeting

Fast image resizing by combining with saliency map and adaptive seam

Some Problems in Seam-Carving Algorithm

Content-aware Deep Perceptual Image Compression

Multi-operator image retargeting with automatic integration of direct and indirect seam carving

Image enlargement by patch-based seam synthesis

Research Progress on Scalable Image Coding

Optimal Bi-Directional Seam Carving for Content-Aware Image Resizing