Lightweight Hybrid Video Compression Framework Using Reference-Guided Restoration Network

Hochang Rhee,Seyun Kim,Nam Ik Cho
2023-03-21
Abstract:Recent deep-learning-based video compression methods brought coding gains over conventional codecs such as AVC and HEVC. However, learning-based codecs generally require considerable computation time and model complexity. In this paper, we propose a new lightweight hybrid video codec consisting of a conventional video codec(HEVC / VVC), a lossless image codec, and our new restoration network. Precisely, our encoder consists of the conventional video encoder and a lossless image encoder, transmitting a lossy-compressed video bitstream along with a losslessly-compressed reference frame. The decoder is constructed with corresponding video/image decoders and a new restoration network, which enhances the compressed video in two-step processes. In the first step, a network trained with a large video dataset restores the details lost by the conventional encoder. Then, we further boost the video quality with the guidance of a reference image, which is a losslessly compressed video frame. The reference image provides video-specific information, which can be utilized to better restore the details of a compressed video. Experimental results show that the proposed method achieves comparable performance to top-tier methods, even when applied to HEVC. Nevertheless, our method has lower complexity, a faster run time, and can be easily integrated into existing conventional codecs.
Image and Video Processing,Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
The problem this paper attempts to address is: how to improve the quality of compressed video while maintaining efficient video compression and reducing the time complexity of encoding and decoding. Specifically, existing deep learning-based video compression methods, although superior in compression performance compared to traditional codecs (such as AVC and HEVC), usually require a significant amount of computation time and model complexity. To solve this problem, the paper proposes a new lightweight hybrid video codec framework that combines traditional video codecs (HEVC or VVC), lossless image codecs, and a new restoration network. Through this combination, the paper aims to achieve the following goals: 1. **Improve the quality of compressed video**: Through a two-step enhancement process, first using a trained network to restore details lost by traditional codecs, and then using reference frames from lossless compression to further enhance video quality. 2. **Reduce the time complexity of encoding and decoding**: Compared to existing deep learning-based methods, this framework significantly reduces the time required for encoding and decoding while maintaining high performance. 3. **Achieve real-time decoding**: By optimizing network structure and algorithm design, the decoding time can reach near real-time levels, making it suitable for practical applications. The paper validates the effectiveness of this method through experiments. The results show that when using HEVC as the baseline, the overall coding gain of this framework can be comparable to the latest top neural codecs, while the required encoding time and complexity are much lower. When combined with VVC, this method can significantly surpass VVC, achieving state-of-the-art coding performance.