Abstract:Deep neural networks (DNN) have been widely applied in many computer vision problems. These tasks are often conducted on input images with high quality without consideration of storage and transmission costs, making it necessary to compress the images for bandwidth-constrained networks. Recently, researches on deep learning based image compression show promising performance compared with traditional image compression codecs. However, these image compression approaches focus on improving user-perceived visual quality, rather achieving high DNN inference accuracy for computer vision. In this work, we design a concrete system with the goal of maximizing the computer vision performance metric, subject to a compression ratio constraint. The entire framework is efficiently optimized in an end-to-end manner without multiple training phases. We find that the conventional distortion metric Mean Squared Error (MSE) in compression does not suffice to get desirable computer vision performance in our system. It is essential to exploit machine-centric evaluation metrics for high inference accuracy. We also propose to apply class-agnostic object masks combined with channel attention mechanism to dynamically allocate bits in regions of interest (ROI) and the background regions (BG) so as to optimize computer vision performance over a range of bitrates. We experiment on three diverse applications separately: image classification, human pose estimation and semantic segmentation. Extensive experiments show that our approach not only outperforms many traditional compression codecs in image compression, but also achieves superior computer vision performance than all other counterparts.

Object-aware Image Compression with Adversarial Learning.

A GAN-based Tunable Image Compression System

Variable Rate ROI Image Compression Optimized for Visual Quality

A Unified Active and Semi-Supervised Learning Framework for Image Compression

Content-aware Deep Perceptual Image Compression

Super-High-Fidelity Image Compression via Hierarchical-ROI and Adaptive Quantization

Learned Low Bit-rate Image Compression with Adversarial Mechanism

Generative Adversarial Networks-Based Image Compression for Consumer Photo Storage.

An Imperceptible Adversarial Attack Against Reconstruction for Learned Image Compression

GAN-based Image Compression with Improved RDO Process

Towards Robust Neural Image Compression: Adversarial Attack and Model Finetuning

Toward Robust Neural Image Compression: Adversarial Attack and Model Finetuning

General Generative Model-Based Image Compression Method Using an Optimisation Encoder.

Subjective Quality Optimized Efficient Image Compression

Region-of-interest and channel attention-based joint optimization of image compression and computer vision

Learning based Facial Image Compression with semantic fidelity metric

Enhancing Adversarial Training with Prior Knowledge Distillation for Robust Image Compression

Coarse-to-Fine Hyper-Prior Modeling for Learned Image Compression

Object-Fidelity Remote Sensing Image Compression With Content-Weighted Bitrate Allocation and Patch-Based Local Attention

Machine Perception-Driven Image Compression: A Layered Generative Approach

Region of Interest Loss for Anonymizing Learned Image Compression