Abstract:Single image novel view synthesis allows the generation of target images with different views from a single input image. Pixel generation methods are one of the main approaches for novel view synthesis, with previous methods typically using the input image to infer the target image in the new view. However, only features from input images in the source view might not be sufficient to generate a good target image, especially when only a single input image is available. In this paper, we fuse features from an input and a warped image to collaboratively generate pixels in the new view, with the warped image as an intermediate output generated by projecting pixels of the input image onto the target view via an estimated depth. Since the estimated depth and the generated warped image are not perfect, errors will be introduced when generating target pixels. To alleviate these and to ensure better channel information between the features from input and warped image, channel attention blocks are employed. In addition, in order to use skip connections for better novel view synthesis results, encoder features in different layers from the input image are transformed to the target view via multi-resolution depths. Here, instead of downsampling a single full-resolution depth to several lower-resolution depths, we adopt a multi-scale depth estimation network to predict multiple depths at different resolutions. Experimental results on benchmark datasets show that our method gives excellent view synthesis results and outperforms other state-of-the-art novel view synthesis methods.

Unsupervised Single-View Synthesis Network via Style Guidance and Prior Distillation

Ivs-Net: Learning Human View Synthesis from Internet Videos

Novel View Synthesis from a Single Unposed Image via Unsupervised Learning

Collaborative Distillation for Ultra-Resolution Universal Style Transfer

Self-Supervised Visibility Learning for Novel View Synthesis

Single-View View Synthesis with Self-Rectified Pseudo-Stereo

Real-Time Unsupervised Multi-View Depth Estimation Network for Virtual View Synthesis

Self-supervised Learning by View Synthesis

View-Aware Geometry-Structure Joint Learning for Single-View 3D Shape Reconstruction

Depth-Assisted Full Resolution Network for Single Image-Based View Synthesis.

AUTO3D: Novel view synthesis through unsupervisely learned variational viewpoint and global 3D representation

Unsupervised virtual view synthesis from monocular video with generative adversarial warping

Novel View Synthesis from only a 6-DoF Camera Pose by Two-stage Networks

Learning By Analogy: Reliable Supervision From Transformations For Unsupervised Optical Flow Estimation

Self-supervised Multi-view Stereo Via Inter and Intra Network Pseudo Depth

Unsupervised multi-view stereo network based on multi-stage depth estimation

A neural refinement network for single image view synthesis

Learning A Locally Unified 3D Point Cloud for View Synthesis

ProbNVS: Fast Novel View Synthesis with Learned Probability-Guided Sampling

Multi-Scale Feature Fusion for Single Image Novel View Synthesis

LVSM: A Large View Synthesis Model with Minimal 3D Inductive Bias