Disparity-Aware Reference Frame Generation Network for Multiview Video Coding
Jianjun Lei,Zongqian Zhang,Zhaoqing Pan,Dong Liu,Xiangrui Liu,Ying Chen,Nam Ling
DOI: https://doi.org/10.1109/tip.2022.3183436
IF: 10.6
2022-07-06
IEEE Transactions on Image Processing
Abstract:Multiview video coding (MVC) aims to compress the multiview video through the elimination of video redundancies, where the quality of the reference frame directly affects the compression efficiency. In this paper, we propose a deep virtual reference frame generation method based on a disparity-aware reference frame generation network (DAG-Net) to transform the disparity relationship between different viewpoints and generate a more reliable reference frame. The proposed DAG-Net consists of a multi-level receptive field module, a disparity-aware alignment module, and a fusion reconstruction module. First, a multi-level receptive field module is designed to enlarge the receptive field, and extract the multi-scale deep features of the temporal and inter-view reference frames. Then, a disparity-aware alignment module is proposed to learn the disparity relationship, and perform disparity shift on the inter-view reference frame to align it with the temporal reference frame. Finally, a fusion reconstruction module is utilized to fuse the complementary information and generate a more reliable virtual reference frame. Experiments demonstrate that the proposed reference frame generation method achieves superior performance for multiview video coding.
computer science, artificial intelligence,engineering, electrical & electronic