Abstract:The past decade has witnessed great success of deep learning technology in many disciplines, especially in computer vision and image processing. However, deep learning-based video coding remains in its infancy. This paper reviews the representative works about using deep learning for image/video coding, which has been an actively developing research area since the year of 2015. We divide the related works into two categories: new coding schemes that are built primarily upon deep networks (deep schemes), and deep network-based coding tools (deep tools) that shall be used within traditional coding schemes or together with traditional coding tools. For deep schemes, pixel probability modeling and auto-encoder are the two approaches, that can be viewed as predictive coding scheme and transform coding scheme, respectively. For deep tools, there have been several proposed techniques using deep learning to perform intra-picture prediction, inter-picture prediction, cross-channel prediction, probability distribution prediction, transform, post- or in-loop filtering, down- and up-sampling, as well as encoding optimizations. In the hope of advocating the research of deep learning-based video coding, we present a case study of our developed prototype video codec, namely Deep Learning Video Coding (DLVC). DLVC features two deep tools that are both based on convolutional neural network (CNN), namely CNN-based in-loop filter (CNN-ILF) and CNN-based block adaptive resolution coding (CNN-BARC). Both tools help improve the compression efficiency by a significant margin. With the two deep tools as well as other non-deep coding tools, DLVC is able to achieve on average 39.6\% and 33.0\% bits saving than HEVC, under random-access and low-delay configurations, respectively. The source code of DLVC has been released for future researches.

Performance Analysis of Deep Learning based on Recurrent Neural Networks for Channel Coding

Performance Evaluation of Channel Decoding with Deep Neural Networks.

DeepTurbo: Deep Turbo Decoder

Decoding 5G-NR Communications via Deep Learning

Doubly Residual Neural Decoder: Towards Low-Complexity High-Performance Channel Decoding

Turbo Autoencoder: Deep learning based channel codes for point-to-point communication channels

Deep Learning Aided Broadcast Codes with Feedback

DeepChannel: Wireless Channel Quality Prediction Using Deep Learning

Recent Advances in Deep Learning for Channel Coding: A Survey

Deep Learning Based Pilot-Free Transmission: Error Correction Coding for Low-Resolution Reception under Time-Varying Channels

Turbo Autoencoder with a Trainable Interleaver

Bidirectional Deep Learning Decoder for Polar Codes in Flat Fading Channels

LEARN Codes: Inventing Low-latency Codes via Recurrent Neural Networks

Deep Learning-Based Video Coding: A Review and A Case Study

Deep Channel Prediction: A DNN Framework for Receiver Design in Time-Varying Fading Channels

Deep Learning for Joint Noncoherent Differential Detection and Channel Decoding.

Coding for the Gaussian Channel in the Finite Blocklength Regime Using a CNN-Autoencoder

Deep Learning for Channel Coding via Neural Mutual Information Estimation

Deep learning-enabled polar code decoders for 5G networks and beyond

Deep Learning for the Degraded Broadcast Channel

DeepIC: Coding for Interference Channels via Deep Learning