Abstract:Deep neural networks (DNN) have been widely applied in many computer vision problems. These tasks are often conducted on input images with high quality without consideration of storage and transmission costs, making it necessary to compress the images for bandwidth-constrained networks. Recently, researches on deep learning based image compression show promising performance compared with traditional image compression codecs. However, these image compression approaches focus on improving user-perceived visual quality, rather achieving high DNN inference accuracy for computer vision. In this work, we design a concrete system with the goal of maximizing the computer vision performance metric, subject to a compression ratio constraint. The entire framework is efficiently optimized in an end-to-end manner without multiple training phases. We find that the conventional distortion metric Mean Squared Error (MSE) in compression does not suffice to get desirable computer vision performance in our system. It is essential to exploit machine-centric evaluation metrics for high inference accuracy. We also propose to apply class-agnostic object masks combined with channel attention mechanism to dynamically allocate bits in regions of interest (ROI) and the background regions (BG) so as to optimize computer vision performance over a range of bitrates. We experiment on three diverse applications separately: image classification, human pose estimation and semantic segmentation. Extensive experiments show that our approach not only outperforms many traditional compression codecs in image compression, but also achieves superior computer vision performance than all other counterparts.

A region-based hierarchical image compression method with simulated visual perception

A GAN-based Tunable Image Compression System

Machine Perception-Driven Image Compression: A Layered Generative Approach

Content-aware Deep Perceptual Image Compression

End-to-end image compression method based on perception metric

A Unified Active and Semi-Supervised Learning Framework for Image Compression

Learned Image Compression for Machine Perception

Exploring Compressed Image Representation as a Perceptual Proxy: A Study

Interpretable Learned Image Compression: A Frequency Transform Decomposition Perspective

Learning Content-Weighted Deep Image Compression

Region-of-interest and channel attention-based joint optimization of image compression and computer vision

Compressive Sensing Image Coding with Perceptual Weighting Measuring Matrix

Learned Block-based Hybrid Image Compression

Research on Application of Perceptual Model Based Image Compression

Semantic Scalable Image Compression with Cross-Layer Priors.

Coarse-to-Fine Hyper-Prior Modeling for Learned Image Compression

HFLIC: Human Friendly Perceptual Learned Image Compression with Reinforced Transform

CPIPS: Learning to Preserve Perceptual Distances in End-to-End Image Compression

Learned Image Compression with Large Capacity and Low Redundancy of Latent Representation

End-to-End Image Compression Via Attention-Guided Information-Preserving Module

Semantics-Guided and Saliency-Focused Learning of Perceptual Video Compression