Abstract:The past decade has witnessed great success of deep learning technology in many disciplines, especially in computer vision and image processing. However, deep learning-based video coding remains in its infancy. This paper reviews the representative works about using deep learning for image/video coding, which has been an actively developing research area since the year of 2015. We divide the related works into two categories: new coding schemes that are built primarily upon deep networks (deep schemes), and deep network-based coding tools (deep tools) that shall be used within traditional coding schemes or together with traditional coding tools. For deep schemes, pixel probability modeling and auto-encoder are the two approaches, that can be viewed as predictive coding scheme and transform coding scheme, respectively. For deep tools, there have been several proposed techniques using deep learning to perform intra-picture prediction, inter-picture prediction, cross-channel prediction, probability distribution prediction, transform, post- or in-loop filtering, down- and up-sampling, as well as encoding optimizations. In the hope of advocating the research of deep learning-based video coding, we present a case study of our developed prototype video codec, namely Deep Learning Video Coding (DLVC). DLVC features two deep tools that are both based on convolutional neural network (CNN), namely CNN-based in-loop filter (CNN-ILF) and CNN-based block adaptive resolution coding (CNN-BARC). Both tools help improve the compression efficiency by a significant margin. With the two deep tools as well as other non-deep coding tools, DLVC is able to achieve on average 39.6\% and 33.0\% bits saving than HEVC, under random-access and low-delay configurations, respectively. The source code of DLVC has been released for future researches.

Current Trends on Deep Learning-aided Channel Coding

Recent Advances in Deep Learning for Channel Coding: A Survey

Deep Learning for Wireless Physical Layer: Opportunities and Challenges

DeepIC: Coding for Interference Channels via Deep Learning

Toward Intelligent Wireless Communications: Deep Learning - Based Physical Layer Technologies

Deep Learning for Wireless Communications: An Emerging Interdisciplinary Paradigm

Deep Learning for Channel Estimation in Physical Layer Wireless Communications: Fundamental, Methods, and Challenges

A Survey on Deep Learning Based Joint Source-Channel Coding

Deep Learning for Channel Coding via Neural Mutual Information Estimation

DeepChannel: Wireless Channel Quality Prediction Using Deep Learning

Performance Analysis of Deep Learning based on Recurrent Neural Networks for Channel Coding

Artificial Intelligence in Optical Communications: From Machine Learning to Deep Learning

Analysis and application research on key issues of channel coding

Deep Learning-Based Video Coding

Model-Driven Deep Learning for Physical Layer Communications

Aider: Artificial Intelligent Based Deep Receiver for Wireless Communication Systems

To Learn or Not to Learn: Deep Learning Assisted Wireless Modem Design

An Introduction to Deep Learning for the Physical Layer

Do not Interfere but Cooperate: A Fully Learnable Code Design for Multi-Access Channels with Feedback

Deep Learning-Based Video Coding: A Review and A Case Study

Deep Learning Aided Broadcast Codes with Feedback