Abstract:Three-dimensional (3D) light field display, as a potential future display method, has attracted considerable attention. However, there still exist certain issues to be addressed, especially the capture of dense views in real 3D scenes. Using sparse cameras associated with view synthesis algorithm has become a practical method. Supervised convolutional neural network (CNN) is used to synthesize virtual views. However, such a large amount of training target views is sometimes difficult to be obtained and the training position is relatively fixed. Novel views can also be synthesized by unsupervised network MPVN, but the method has strict requirements on capturing multiple uniform horizontal viewpoints, which is not suitable in practice. Here, a method of dense-view synthesis based on unsupervised learning is presented, which can synthesize arbitrary virtual views with multiple free-posed views captured in the real 3D scene based on unsupervised learning. Multiple posed views are reprojected to the target position and input into the neural network. The network outputs a color tower and a selection tower indicating the scene distribution along the depth direction. A single image is yielded by the weighted summation of two towers. The proposed network is end-to-end trained based on unsupervised learning by minimizing errors during reconstructions of posed views. A virtual view can be predicted in a high quality by reprojecting posed views to the desired position. Additionally, a sequence of dense virtual views can be generated for 3D light-field display by repeated predictions. Experimental results demonstrate the validity of our proposed network. PSNR of synthesized views are around 30dB and SSIM are over 0.90. Since multiple cameras are supported to be placed in free-posed positions, there are not strict physical requirements and the proposed method can be flexibly used for the real scene capture. We believe this approach will contribute to the wide applications of 3D light-field display in the future.

DCM-CNN: Densely Connected Multiloss Convolutional Neural Networks for Light Field View Synthesis

High-Quality Virtual View Synthesis for Light Field Cameras Using Multi-Loss Convolutional Neural Networks

A Learning-Based Method Using Epipolar Geometry for Light Field Depth Estimation

Light-field view synthesis using convolutional block attention module

Dense view synthesis for three-dimensional light-field displays based on position-guiding convolutional neural network

View Position Prior-Supervised Light Field Angular Super-Resolution Network with Asymmetric Feature Extraction and Spatial-Angular Interaction.

Depth-guided View Synthesis for Light Field Reconstruction from a Single Image

Low-complexity ℓ∞-Compression of Light Field Images with a Deep-Decompression Stage

Unsupervised Dense Light Field Reconstruction with Occlusion Awareness

Dense reconstruction for sparsely-sampled light field

A Coarse-to-Fine Convolutional Neural Network for Light Field Angular Super-Resolution

Multi-parallax views synthesis for three-dimensional light-field display using unsupervised CNN.

Light Field Reconstruction Using Convolutional Network on EPI and Extended Applications

Light Field Image Super-Resolution Network Based on Angular Difference Enhancement

Breaking the Spatio-Angular Trade-off for Light Field Super-Resolution via LSTM Modelling on Epipolar Plane Images.

Dense-view Synthesis for Three-Dimensional Light-Field Display Based on Unsupervised Learning.

Light-field-depth-estimation Network Based on Epipolar Geometry and Image Segmentation.

Unsupervised Depth Estimation from Light Field Using a Convolutional Neural Network

A Fast View Synthesis Implementation Method for Light Field Applications.

High-Dimensional Dense Residual Convolutional Neural Network for Light Field Reconstruction

A Learning-based Method for Conditioning Neural Light Fields from Limited Inputs