Abstract:Deep neural networks (DNN) have been widely applied in many computer vision problems. These tasks are often conducted on input images with high quality without consideration of storage and transmission costs, making it necessary to compress the images for bandwidth-constrained networks. Recently, researches on deep learning based image compression show promising performance compared with traditional image compression codecs. However, these image compression approaches focus on improving user-perceived visual quality, rather achieving high DNN inference accuracy for computer vision. In this work, we design a concrete system with the goal of maximizing the computer vision performance metric, subject to a compression ratio constraint. The entire framework is efficiently optimized in an end-to-end manner without multiple training phases. We find that the conventional distortion metric Mean Squared Error (MSE) in compression does not suffice to get desirable computer vision performance in our system. It is essential to exploit machine-centric evaluation metrics for high inference accuracy. We also propose to apply class-agnostic object masks combined with channel attention mechanism to dynamically allocate bits in regions of interest (ROI) and the background regions (BG) so as to optimize computer vision performance over a range of bitrates. We experiment on three diverse applications separately: image classification, human pose estimation and semantic segmentation. Extensive experiments show that our approach not only outperforms many traditional compression codecs in image compression, but also achieves superior computer vision performance than all other counterparts.

Region-Based Image-Fusion Framework for Compressive Imaging

Classification-based image-fusion framework for compressive imaging

Dual Band Image Fusion Method Based on Region Saliency

Region-aware RGB and Near-Infrared Image Fusion

Multi-Frame Image Fusion Method Combining Spatial-Temporal Saliency Detection and Nsct

PCNN-Based Image Fusion in Compressed Domain

Gradient-based Compressive Image Fusion

A Framework of Region-Based Dynamic Image Fusion

Image fusion in compressed sensing

Region-of-interest and channel attention-based joint optimization of image compression and computer vision

A Regional Image Fusion Based on Similarity Characteristics

Compression Ratio Learning and Semantic Communications for Video Imaging

FIPER: Generalizable Factorized Fields for Joint Image Compression and Super-Resolution

Integration of Communication and Computational Imaging

A Unified Framework for Jointly Compressing Visual and Semantic Data

CHFusion: A Cross-modality High-resolution Representation Framework for Infrared and Visible Image Fusion

SIFusion: Lightweight infrared and visible image fusion based on semantic injection

Infrared and visible image fusion method based on visual saliency objects and fuzzy region attributes

CoCoCs: Co-Optimized Compressive Imaging Driven by High-Level Vision

Image-Compression Techniques: Classical and "Region-of-Interest-Based" Approaches Presented in Recent Papers

Unified and Scalable Deep Image Compression Framework for Human and Machine