Abstract:Inspired by the recent advances of image super-resolution using convolutional neural network (CNN), we propose a CNN-based block up-sampling scheme for intra frame coding. A block can be down-sampled before being compressed by normal intra coding, and then up-sampled to its original resolution. Different from previous studies on down/up-sampling-based coding, the up-sampling methods in our scheme have been designed by training CNN instead of hand-crafted. We explore a new CNN structure for up-sampling, which features deconvolution of feature maps, multi-scale fusion, and residue learning, making the network both compact and efficient. We also design different networks for the up-sampling of luma and chroma components, respectively, where the chroma up-sampling CNN utilizes the luma information to boost its performance. In addition, we design a two-stage up-sampling process, the first stage being within the block-by-block coding loop, and the second stage being performed on the entire frame, so as to refine block boundaries. We also empirically study how to set the coding parameters of down-sampled blocks for pursuing the frame-level rate-distortion optimization. Our proposed scheme is implemented into the high-efficiency video coding (HEVC) reference software, and a comprehensive set of experiments have been performed to evaluate our methods. Experimental results show that our scheme achieves significant bits saving compared with the HEVC anchor, especially at low bit rates, leading to on average 5.5% BD-rate reduction on common test sequences and on average 9.0% BD-rate reduction on ultrahigh definition test sequences.

Combining Intra Block Copy and Neighboring Samples Using Convolutional Neural Network for Image Coding

Fully Connected Network-Based Intra Prediction for Image Coding.

Intra Block Partition Structure Prediction Via Convolutional Neural Network.

Intra Prediction Using Fully Connected Network for Video Coding

Spatio-Temporal Convolutional Neural Network for Enhanced Inter Prediction in Video Coding

Enhanced Bi-Prediction with Convolutional Neural Network for High-Efficiency Video Coding

Optimized Spatial Recurrent Network for Intra Prediction in Video Coding

Convolutional Neural Network-Based Block Up-Sampling for Intra Frame Coding

Multi-Scale Convolutional Neural Network-Based Intra Prediction for Video Coding.

Enhanced Intra Prediction with Recurrent Neural Network in Video Coding

Neural Network-Based Enhancement to Inter Prediction for Video Coding

Deep Convolutional Neural Network For Decompressed Video Enhancement

Local-Aware Intra Block Copy for Video Coding.

A Prediction-Transformation-combined Intra Picture Coding Method.

Hierarchical Piece-Wise Linear Projections for Efficient Intra-Prediction Coding.

Learned Block-based Hybrid Image Compression

NR-CNN: Nested-Residual Guided CNN In-loop Filtering for Video Coding

Faster Intra-Prediction of Versatile Video Coding Using a Concatenate-Designed CNN via DCT Coefficients

Dual Learning-based Video Coding with Inception Dense Blocks

A Hybrid Neural Network for Chroma Intra Prediction

Combining Directional Intra Prediction And Intra Block Copy With Block Partition For Hevc