Abstract:Image coding is one of the most fundamental techniques and is widely used in image/video processing and multimedia communications. Current image coding methods are mainly human-oriented, and the visual quality is always unsatisfactory, especially at low bitrates. Moreover, the recent emergence of machine vision goes beyond the scope of current coding. With these considerations, we proposed a sketch assisted face image coding for human and machine vision by a joint training approach. In the proposed approach, we design a new feature representation: a color sketch, which aims to satisfy both low-frequency features of human vision and high-frequency features of machine analysis. Then, we present a novel end-to-end image codec framework with joint training that consists of three models: an image-to-image translation module, a coding module, and a two-stage reconstruction module. Specifically, the input image is first translated into the edge map with the Canny edge as the auxiliary label to merely preserve the structure information. Afterward, the backpropagation from reconstruction module guides the edge map to increase or decrease the information through joint training, which results in the generation of color sketch. Then, the generated sketch is compressed into the bitstream and decompressed back to a sketch in the coding module. Finally, the decompressed sketch is reconstructed to support the machine and human tasks, respectively. In this way, the color sketch is designed to bridge the gap between human and machine vision, and the joint training strategy helps to adjust the low-frequency information in the sketch. The experimental results on challenge datasets demonstrate that our proposed algorithm offers 40.9%-86.6% bitrate savings on machine vision and is comparable to state-of-the-art image coding methods on human vision.

Joint super-resolution-based fast face image coding for human and machine vision

Foveated JND Model Based on Stereo Vision and Its Application in Image Compression with Hardware Implementation

Boosting High-Level Vision with Joint Compression Artifacts Reduction and Super-Resolution

Towards Coding for Human and Machine Vision: Scalable Face Image Coding

Group Image Compression for Dual Use of Machine and Human Vision

Sketch Assisted Face Image Coding for Human and Machine Vision: A Joint Training Approach

A Deep Image Compression Framework for Face Recognition

Deep Image Compression Towards Machine Vision: A Unified Optimization Framework

Deep Image Compression Toward Machine Vision: A Unified Optimization Framework

The face image super-resolution algorithm based on combined representation learning

Joint Rate-Distortion Optimization For Simultaneous Texture And Deep Feature Compression Of Facial Images

Towards On-demand Transmission: Joint Feature and Image Coding with Reversible Neural Networks

Region-of-interest and channel attention-based joint optimization of image compression and computer vision

Scalable image coding with enhancement features for human and machine

Slimmable Multi-Task Image Compression for Human and Machine Vision

Content-aware Facial Image Compression with Deep Learning Method

Collaborative Scalable Visual Compression for Human-Centered Videos.

Scalable Face Image Coding via StyleGAN Prior: Towards Compression for Human-Machine Collaborative Vision

Scalable Face Image Coding via StyleGAN Prior: Toward Compression for Human-Machine Collaborative Vision

DeepN-JPEG: A Deep Neural Network Favorable JPEG-based Image Compression Framework

JsrNet: A Joint Sampling-Reconstruction Framework for Distributed Compressive Video Sensing