Improving Statistical Fidelity for Neural Image Compression with Implicit Local Likelihood Models

Matthew J. Muckley,Alaaeldin El-Nouby,Karen Ullrich,Hervé Jégou,Jakob Verbeek

2023-08-11

Abstract:Lossy image compression aims to represent images in as few bits as possible while maintaining fidelity to the original. Theoretical results indicate that optimizing distortion metrics such as PSNR or MS-SSIM necessarily leads to a discrepancy in the statistics of original images from those of reconstructions, in particular at low bitrates, often manifested by the blurring of the compressed images. Previous work has leveraged adversarial discriminators to improve statistical fidelity. Yet these binary discriminators adopted from generative modeling tasks may not be ideal for image compression. In this paper, we introduce a non-binary discriminator that is conditioned on quantized local image representations obtained via VQ-VAE autoencoders. Our evaluations on the CLIC2020, DIV2K and Kodak datasets show that our discriminator is more effective for jointly optimizing distortion (e.g., PSNR) and statistical fidelity (e.g., FID) than the PatchGAN of the state-of-the-art HiFiC model. On CLIC2020, we obtain the same FID as HiFiC with 30-40\% fewer bits.

Image and Video Processing,Artificial Intelligence,Computer Vision and Pattern Recognition,Information Theory

What problem does this paper attempt to address?

The paper attempts to address the trade-off between statistical fidelity and distortion in neural image compression at low bit rates. Specifically, existing methods, when optimizing distortion metrics (such as PSNR or MS-SSIM), result in differences in the statistical properties between the compressed image and the original image. This difference often manifests as blurriness in the compressed image, especially at low bit rates. Although previous work has utilized discriminators in Generative Adversarial Networks (GANs) to improve statistical fidelity, these binary discriminators may not be suitable for image compression tasks.

To address this issue, this paper proposes a new non-binary discriminator based on quantized local image representations (obtained through a VQ-VAE autoencoder). Experimental results show that this discriminator is more effective than the PatchGAN discriminator in the HiFiC model for jointly optimizing distortion (such as PSNR) and statistical fidelity (such as FID). On the CLIC2020 dataset, this method can achieve the same FID score as HiFiC while reducing the bit rate by 30-40%.