Abstract:View synthesis aims to produce unseen views from a set of views captured by two or more cameras at different positions. This task is non-trivial since it is hard to conduct pixel-level matching among different views. To address this issue, most existing methods seek to exploit the geometric information to match pixels. However, when the distinct cameras have a large baseline (i. e., far away from each other), severe geometry distortion issues would occur and the geometric information may fail to provide useful guidance, resulting in very blurry synthesized images. To address the above issues, in this paper, we propose a novel deep generative model, called Self-Consistent Generative Network (SCGN), which synthesizes novel views from the given input views without explicitly exploiting the geometric information. The proposed SCGN model consists of two main components, i. e., a View Synthesis Network (VSN) and a View Decomposition Network (VDN), both employing an Encoder-Decoder structure. Here, the VDN seeks to reconstruct input views from the synthesized novel view to preserve the consistency of view synthesis. Thanks to VDN, SCGN is able to synthesize novel views without using any geometric rectification before encoding, making it easier for both training and applications. Finally, adversarial loss is introduced to improve the photo-realism of novel views. Both qualitative and quantitative comparisons against several state-of-the-art methods on two benchmark tasks demonstrated the superiority of our approach.

Generalizable Novel-View Synthesis using a Stereo Camera

Unifying Correspondence, Pose and NeRF for Pose-Free Novel View Synthesis from Stereo Pairs

Unifying Correspondence, Pose and NeRF for Generalized Pose-Free Novel View Synthesis

Novel View Synthesis from only a 6-DoF Camera Pose by Two-stage Networks

Single-View View Synthesis with Self-Rectified Pseudo-Stereo

Deep View Synthesis Via Self-Consistent Generative Network

Multi-View To Novel View: Synthesizing Novel Views With Self-Learned Confidence

G-NeRF: Geometry-enhanced Novel View Synthesis from Single-View Images

Fast Generalizable Novel View Synthesis with Uncertainty-Aware Sampling

Novel View Synthesis with View-Dependent Effects from a Single Image

AUTO3D: Novel view synthesis through unsupervisely learned variational viewpoint and global 3D representation

EpipolarNVS: leveraging on Epipolar geometry for single-image Novel View Synthesis

View Synthesis with Multi-scale Cost Aggregation and Confidence Prior

Novel View Synthesis from a Single RGBD Image for Indoor Scenes

Multi-View Stereo and Depth Priors Guided NeRF for View Synthesis.

Consistent-1-to-3: Consistent Image to 3D View Synthesis via Geometry-aware Diffusion Models

ContraNeRF: Generalizable Neural Radiance Fields for Synthetic-to-real Novel View Synthesis via Contrastive Learning

ViVid-1-to-3: Novel View Synthesis with Video Diffusion Models

ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis

Free3D: Consistent Novel View Synthesis without 3D Representation

Pseudo-Generalized Dynamic View Synthesis from a Video