Joint depth video enhancement and fast intra encoding algorithm in 3D-HEVC
Chao Huang,Zongju Peng,Jinchao Miao,Fen Chen
DOI: https://doi.org/10.11834/jig.170452
2018-01-01
Journal of Image and Graphics
Abstract:Objective With the development of 3D content acquisition and display technologies in recent years,three-dimensional (3D) video has received increasing attention.The multi-view video plus depth format is the main representation of a 3D scene.In the 3D extension of high-efficiency video coding (3D-HEVC),the main framework for depth video is similar to that of HEVC.Each coding unit (CU) is recursively divided into four sub-CUs.Each CU depth level enables 37 types of intra modes in intra frames.Unlike conventional texture videos,depth videos are not used for watching,but for virtual view rendering.The preservation of depth sharp object edges is important for depth video compression.Several new techniques,such as depth modeling mode (DMM) and view synthesis optimization,are introduced into the current 3 D-HEVC test model to improve the efficiency of depth video intra coding and the quality of synthesized views.These techniques improve the coding efficiency of depth videos.However,they greatly increase the computational complexity of depth intra coding,thereby hindering the real-time applications of 3D-HEVC.Depth videos are also inaccurate and inconsistent because of the limitations of mainstream capture technologies.The inaccuracy of depth videos further increases the computational complexity of intra coding.Previous research on low-complexity depth video intra coding and depth video enhancement has been conducted separately.Thus,a joint depth video enhancement and fast intra-coding algorithm is proposed in this study.Method An enhancement method is applied before encoding to remove inaccurate textures in a depth video and enhance the spatial correlation of the depth video.The edge region is preserved for rendering performance.For non-edge regions,Gaussian and adaptive window smoothing filters are used.The enhanced depth video is mainly characterized by sharp object edges and large areas of nearly constant regions.We can skip some prediction modes and CU depth levels rarely used in homogeneity regions by fully exploiting such features.CUs are classified according to texture complexity,and the partition process of CUs with low texture complexity is terminated early.Prediction units (PUs) are classified according to edge intensity.The proposed algorithm selectively omits unnecessary DMM in the mode decision process on the basis of the PU classification results.The algorithm is implemented on the reference software,HTM-16.0,of the 3D-HEVC standard and tested under common test conditions required by the Joint Collaborative Team on 3D Video Coding to evaluate its performance.The proposed algorithm is specially designed for depth videos that are estimated using stereo matching;thus,sequences synthesized by a computer are not tested.The proposed scheme aims at depth video intra coding,and all test sequences are coded with intra-only structure and three-view configuration.The rate distortion performance of the proposed algorithm is evaluated by using the Bjontegaard delta bitrate,which is calculated by the peak signal-to-noise ratio of the synthesis view quality and the total bitrate,including color and depth videos.Result Experimental results show that the proposed algorithm significantly saves the encoding time of the depth video and reduces bitrate under the same synthesized virtual view quality.The coding time reduction obtained by the proposed algorithm compared with that of the original 3D-HEVC encoder ranges from 61.35％ to 65.73％ and is 62.91％ on average.In terms of coding efficiency,our proposed algorithm can reduce bitrate by 4.63％ under the condition of the same synthesized virtual view quality,in which the maximal and minimal reductions are 8.10％ and 2.60％,respectively.The subjective quality of the proposed algorithm is significantly improved compared with that of the original 3D-HEVC encoder.The significant performance improvement of depth video coding contributes to the depth video enhancement and the fast algorithm.The proposed algorithm is superior to the state-of-the-art fast depth intra-coding algorithm.The encoding time saving of depth video is greatly increased by 26.10％,and the bitrate is further reduced by 5.20％ under the condition of the same synthesized virtual view quality.Conclusion A joint depth video enhancement and fast intra-coding algorithm is proposed to solve the problems in 3D-HEVC,the high computational complexity of depth video intra coding,and the inaccuracy of depth videos.The proposed enhancement method improves the spatial correlation of depth videos.The fast intra-coding scheme can significantly reduce the encoding time of depth videos.Therefore,the proposed method not only reduces encoding time but also improves the compression performance of depth video intra coding.