Abstract:Deep neural networks (DNN) have been widely applied in many computer vision problems. These tasks are often conducted on input images with high quality without consideration of storage and transmission costs, making it necessary to compress the images for bandwidth-constrained networks. Recently, researches on deep learning based image compression show promising performance compared with traditional image compression codecs. However, these image compression approaches focus on improving user-perceived visual quality, rather achieving high DNN inference accuracy for computer vision. In this work, we design a concrete system with the goal of maximizing the computer vision performance metric, subject to a compression ratio constraint. The entire framework is efficiently optimized in an end-to-end manner without multiple training phases. We find that the conventional distortion metric Mean Squared Error (MSE) in compression does not suffice to get desirable computer vision performance in our system. It is essential to exploit machine-centric evaluation metrics for high inference accuracy. We also propose to apply class-agnostic object masks combined with channel attention mechanism to dynamically allocate bits in regions of interest (ROI) and the background regions (BG) so as to optimize computer vision performance over a range of bitrates. We experiment on three diverse applications separately: image classification, human pose estimation and semantic segmentation. Extensive experiments show that our approach not only outperforms many traditional compression codecs in image compression, but also achieves superior computer vision performance than all other counterparts.

Manipulating image patches for compression

Inpainting with image patches for compression

Image Compression with Edge-Based Inpainting

Learned Distributed Image Compression with Multi-Scale Patch Matching in Feature Domain

Compression with Vision Technologies

Sparse Representation of Texture Patches for Low Bit-Rate Image Compression

CPIPS: Learning to Preserve Perceptual Distances in End-to-End Image Compression

Correlation Estimation from Compressed Images

Region-of-interest and channel attention-based joint optimization of image compression and computer vision

Block-based Image Compression with Parameter-Assistant Inpainting

Machine Perception-Driven Image Compression: A Layered Generative Approach

Padding-Aware Learned Image Compression.

Three-Dimensional Point-Cloud Plus Patches: Towards Model-Based Image Coding in the Cloud

Compressed Image Restoration via External-Image Assisted Band Adaptive PCA Model Learning

Coarse-to-Fine Hyper-Prior Modeling for Learned Image Compression

Content-aware Deep Perceptual Image Compression

Flexible Coding Order for Learned Image Compression

Compression Noise Estimation and Reduction Via Patch Clustering

Perceptual Image Compression with Cooperative Cross-Modal Side Information

SIFT-Based Image Compression

Image Compression With Structure-Aware Inpainting