Abstract:In the field of neural data compression, the prevailing focus has been on optimizing algorithms for either classical distortion metrics, such as PSNR or SSIM, or human perceptual quality. With increasing amounts of data consumed by machines rather than humans, a new paradigm of machine-oriented compression$\unicode{x2013}$which prioritizes the retention of features salient for machine perception over traditional human-centric criteria$\unicode{x2013}$has emerged, creating several new challenges to the development, evaluation, and deployment of systems utilizing lossy compression. In particular, it is unclear how different approaches to lossy compression will affect the performance of downstream machine perception tasks. To address this under-explored area, we evaluate various perception models$\unicode{x2013}$including image classification, image segmentation, speech recognition, and music source separation$\unicode{x2013}$under severe lossy compression. We utilize several popular codecs spanning conventional, neural, and generative compression architectures. Our results indicate three key findings: (1) using generative compression, it is feasible to leverage highly compressed data while incurring a negligible impact on machine perceptual quality; (2) machine perceptual quality correlates strongly with deep similarity metrics, indicating a crucial role of these metrics in the development of machine-oriented codecs; and (3) using lossy compressed datasets, (e.g. ImageNet) for pre-training can lead to counter-intuitive scenarios where lossy compression increases machine perceptual quality rather than degrading it. To encourage engagement on this growing area of research, our code and experiments are available at:

Research on Application of Perceptual Model Based Image Compression

Compressive Sensing Image Coding with Perceptual Weighting Measuring Matrix

End-to-end image compression method based on perception metric

A region-based hierarchical image compression method with simulated visual perception

Interpretable Learned Image Compression: A Frequency Transform Decomposition Perspective

A Survey on Perceptually Optimized Video Coding

Good, Cheap, and Fast: Overfitted Image Compression with Wasserstein Distortion

Towards image compression with perfect realism at ultra-low bitrates

Efficient Video Coding Optimization Using a Novel Perceptual Distortion Model

Low-bit-rate wavelet-based image compression using human visual characteristics

Machine Perceptual Quality: Evaluating the Impact of Severe Lossy Compression on Audio and Image Models

Content-aware Deep Perceptual Image Compression

Perceptual Quality-Oriented Rate Allocation via Distillation from End-to-End Image Compression

Machine Perception-Driven Image Compression: A Layered Generative Approach

Learned Image Compression for Machine Perception

Visual Analysis Motivated Rate-Distortion Model for Image Coding

A Video Image Compression Method Based on Visually Salient Features.

Perceptually Optimizing Deep Image Compression

Traditional Transformation Theory Guided Model for Learned Image Compression

Consistency Guided Diffusion Model with Neural Syntax for Perceptual Image Compression

CPIPS: Learning to Preserve Perceptual Distances in End-to-End Image Compression