A Deep Image Coding Scheme With Generative Network to Learn From Correlated Images

Yihao Chen,Bin Tan,Jun Wu,Zhifeng Zhang,Haoqi Ren

DOI: https://doi.org/10.1109/tmm.2021.3087011

IF: 7.3

2021-01-01

IEEE Transactions on Multimedia

Abstract:This paper provides a method to build a deep learning image coding system based on inverse problem, choosing a suitable measurement operator to reduce the amount of information transmitted at the sender, and reconstructing the original image by tackling the inverse problem at the receiver. Unlike most compressed sensing (CS) methods, the proposed coding scheme does not rely on sparsity but uses the structural priors of the generative adversarial networks (GAN) to solve the inverse problem. The proposed model trains the GAN to learn a mapping from the latent space to the sample space formed by correlated images on the cloud. Then the measurements are used to localize the optimal latent variable in the representation space which corresponding to the original image in the sample space. The proposed method encodes and transmits the measurements instead of the original image, which greatly reduces the cost of transmission while ensuring the quality of the reconstructed the image at high compression ratios. To the best of our knowledge, this is the first time to introduce the GAN-based inverse problem in the field of the deep image coding area. The experimental results show that the visual quality of the images generated by the proposed scheme is better than the traditional encoding scheme JPEG2000. Especially in the case of extremely high compression ratios, the proposed scheme can still maintain good performance.

computer science, information systems,telecommunications, software engineering

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to use Generative Adversarial Networks (GAN) in image encoding to reduce transmission costs while maintaining high - quality images. Specifically, the paper proposes a deep - image - encoding scheme based on inverse problems. This scheme uses GAN to learn from related images and reconstructs the original image at the receiving end by solving the inverse problem. Different from traditional compressed - sensing methods, this method does not rely on the sparsity of images but uses the structural prior of GAN to solve the inverse problem. The main contribution of the paper lies in introducing GAN into the field of deep - image - encoding for the first time and proposing an asymmetric encoding scheme, that is, only deploying a deep neural network at the receiving end, thus solving the problem of hardware limitations at the sending end. In addition, experimental results show that in the case of a high compression ratio, the proposed scheme is superior to the traditional JPEG2000 encoding scheme in terms of visual quality.