Abstract:Emerging Learned image Compression (LC) achieves significant improvements in coding efficiency by end-to-end training of neural networks for compression. An important benefit of this approach over traditional codecs is that any optimization criteria can be directly applied to the encoder-decoder networks during training. Perceptual optimization of LC to comply with the Human Visual System (HVS) is among such criteria, which has not been fully explored yet. This paper addresses this gap by proposing a novel framework to integrate Just Noticeable Distortion (JND) principles into LC. Leveraging existing JND datasets, three perceptual optimization methods are proposed to integrate JND into the LC training process: (1) Pixel-Wise JND Loss (PWL) prioritizes pixel-by-pixel fidelity in reproducing JND characteristics, (2) Image-Wise JND Loss (IWL) emphasizes on overall imperceptible degradation levels, and (3) Feature-Wise JND Loss (FWL) aligns the reconstructed image features with perceptually significant features. Experimental evaluations demonstrate the effectiveness of JND integration, highlighting improvements in rate-distortion performance and visual quality, compared to baseline methods. The proposed methods add no extra complexity after training.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: in Learned Image Compression (LC) based on deep learning, how to introduce visual perception optimization (especially the Just Noticeable Distortion (JND) principle) to improve compression efficiency and the visual quality of reconstructed images. Specifically, the paper aims to make up for the deficiencies of existing LC methods in perception optimization, especially the under - explored application of the JND principle. ### Problem Background Traditional image compression methods (such as JPEG, HEVC, etc.) have made significant progress in efficiency, but they usually rely on fixed encoding and decoding processes and are difficult to adapt flexibly to different optimization goals. In contrast, Learned Image Compression (LC) methods based on deep learning can optimize compression performance by training neural networks end - to - end. An important advantage of this method is that any optimization criteria can be directly applied, including perception optimization consistent with the Human Visual System (HVS). However, although LC methods can be flexibly optimized, the current research on perception optimization is not in - depth enough, especially on the problem of how to effectively integrate the JND principle into the LC framework. The JND principle refers to the minimum distortion level that the human visual system can detect. Using this principle can guide the compression algorithm to maximize compression efficiency without affecting visual perception. ### Core Problems of the Paper The main goal of the paper is to design a new framework that integrates the JND principle into the LC training process to achieve more efficient compression and better visual quality. Specifically, the paper raises the following questions: 1. **How to effectively introduce the JND principle in LC?** - The paper proposes a JND - based optimization framework and designs three loss functions (Pixel - Wise JND Loss, Image - Wise JND Loss, Feature - Wise JND Loss) to guide the network to learn image representations that conform to JND characteristics. 2. **Can these new methods improve compression efficiency and visual quality?** - Through experimental verification, the paper shows the improvements of the new methods in rate - distortion performance and visual quality, especially more obvious at low bit rates. 3. **How to balance perception optimization and model generalization ability?** - The paper explores how to combine large - scale unlabeled data sets and limited JND - labeled data sets for training to avoid over - fitting and ensure the generalization ability of the model. ### Summary The core problem of this paper is to design a new perception - optimized framework by introducing the JND principle to improve the efficiency and visual quality of deep - learning - based image compression methods. The paper not only proposes specific solutions but also verifies the effectiveness of these methods through extensive experiments.

Perceptual Learned Image Compression via End-to-End JND-Based Optimization

Foveated JND Model Based on Stereo Vision and Its Application in Image Compression with Hardware Implementation

Channel-Wise Feature Decorrelation for Enhanced Learned Image Compression

Learned Image Compression for Machine Perception

Multiscale Progressive Image Compression Network Guided by Learnable Just Noticeable Distortion

HFLIC: Human Friendly Perceptual Learned Image Compression with Reinforced Transform

Hierarchical Predictive Coding-Based JND Estimation for Image Compression

End-to-End Learnt Image Compression via Non-Local Attention Optimization and Improved Context Modeling

Efficient Learned Lossless JPEG Recompression

High Visual-Fidelity Learned Video Compression

Deep Perceptual Compression

Practical Full Resolution Learned Lossless Image Compression

Perceptually Optimizing Deep Image Compression

Learned Image Compression With Gaussian-Laplacian-Logistic Mixture Model and Concatenated Residual Modules

CPIPS: Learning to Preserve Perceptual Distances in End-to-End Image Compression

Learned Lossless Compression for JPEG via Frequency-Domain Prediction

Perceptual Learned Source-Channel Coding for High-Fidelity Image Semantic Transmission

Asymmetric Learned Image Compression with Multi-Scale Residual Block, Importance Scaling, and Post-Quantization Filtering

Deep Image Compression via End-to-End Learning

A region-based hierarchical image compression method with simulated visual perception