Adversarial Distortion for Learned Video Compression

Vijay Veerabadran,Reza Pourreza,Amirhossein Habibian,Taco Cohen

DOI: https://doi.org/10.48550/arXiv.2004.09508

2021-06-19

Abstract:In this paper, we present a novel adversarial lossy video compression model. At extremely low bit-rates, standard video coding schemes suffer from unpleasant reconstruction artifacts such as blocking, ringing etc. Existing learned neural approaches to video compression have achieved reasonable success on reducing the bit-rate for efficient transmission and reduce the impact of artifacts to an extent. However, they still tend to produce blurred results under extreme compression. In this paper, we present a deep adversarial learned video compression model that minimizes an auxiliary adversarial distortion objective. We find this adversarial objective to correlate better with human perceptual quality judgement relative to traditional quality metrics such as MS-SSIM and PSNR. Our experiments using a state-of-the-art learned video compression system demonstrate a reduction of perceptual artifacts and reconstruction of detail lost especially under extremely high compression.

Image and Video Processing,Computer Vision and Pattern Recognition,Machine Learning

What problem does this paper attempt to address?

This paper attempts to solve the problem of unpleasant reconstruction artifacts (such as blocking effect, ringing effect, etc.) in video compression at very low bit rates. Although the existing neural - network - based learned video compression methods have achieved certain success in reducing the bit rate for efficient transmission and have reduced the influence of artifacts to a certain extent, they still tend to produce blurry results under extreme compression conditions. For this reason, the paper proposes a new video compression model with adversarial loss, which improves the perceptual quality by minimizing the auxiliary adversarial distortion objective. Experiments show that this model can reduce perceptual artifacts and restore details under extremely high compression conditions.

Specifically, the main contributions of the paper include:
1. Proposing the use of adversarial loss to improve the perceptual quality of learned video compression.
2. Studying the techniques of using adversarial loss for training to improve the stability of training.
3. Studying the method of spatio - temporal decomposition discriminator to support the end - to - end training of deep video compression networks.

These improvements aim to make the video compression system improve the visual quality of videos while maintaining a low bit rate, especially under extreme compression conditions.