Abstract:Depth image-based rendering (DIBR) is an important technology in the process of 2D-to-3D conversion. It uses texture images and related depth maps to render virtual views. While there are still some challenging problems in the current DIBR systems, such as disocclusion occurrences. Inpainting methods based on deep learning have recently shown significant improvements and generated plausible images. However, most of these methods may not deal well with the disocclusion holes in the synthesized views, because on the one hand they only treat this issue as generative inpainting after 3D warping, rather than following the full DIBR processing procedures. While on the other hand the distributions of holes on the virtual views are always around the transition regions of foreground and background, which makes them more difficult to distinguish without special constraints. Motivated by these observations, this paper proposes a novel learning-based method for stereoscopic view synthesis, in which the disocclusion regions are restored by a progressive structure reconstruction strategy instead of direct texture inpainting. Additionally, some special cues in the synthesized scenes are further exploited as constraints for the network to alleviate hallucinated structure mixtures among different layers. Extensive empirical evaluations and comparisons validate the strengths of the proposed approach and demonstrate that the model is more suitable for stereoscopic synthesis in the 2D-to-3D conversion applications.

A Novel Filling Disocclusion Method Based on Background Extraction in Depth-Image-Based-Rendering

A New Method of DIBR Based on Background Inpainting

A New Disocclusion Filling Approach in Depth Image Based Rendering for Stereoscopic Imaging

Depth-Aided Inpainting for Disocclusion Restoration of Multi-View Images Using Depth-Image-Based Rendering

A Novel DIBR Method Based on Image Inpainting

A NEW DEPTH-AIDED MULTIDIRECTIONAL DISOCCLUSION RESTORATION METHOD FOR DEPTH-IMAGE-BASED RENDERING

A New Multidirectional Extrapolation Hole-Filling Method For Depth-Image-Based Rendering

Hole-filling based on disparity map for DIBR.

Depth-image-based Rendering with Spatial and Temporal Texture Synthesis for 3DTV

Arbitrary view generation based on DIBR

Hierarchical Hole Filling for Novel View Synthesis

A novel 2D-to-3D conversion method based on blocks world

A New Virtual View Synthesis Method Using SLIC-based Hole Filling

A Novel Method of Multi-view Virtual Image Synthesis for Auto-stereoscopic Display

Real-time Depth Extraction and Multi-View Rendering Algorithm Based on Kinect

A Hole Filling Approach Based on Background Reconstruction for View Synthesis in 3d Video

A Disocclusion Inpainting Framework for Depth-Based View Synthesis

Arbitrary View Rendering Based on DIBR and Image Mergence

Stereoscopic view synthesis with progressive structure reconstruction and scene constraints

A New Virtual View Rendering Method Based on Depth Image

Adaptive 3D content generation via depth video based rendering