Abstract:Novel view synthesis is a long-standing problem that revolves around rendering frames of scenes from novel camera viewpoints. Volumetric approaches provide a solution for modeling occlusions through the explicit 3D representation of the camera frustum. Multi-plane Images (MPI) are volumetric methods that represent the scene using front-parallel planes at distinct depths but suffer from depth discretization leading to a 2.D scene representation. Another line of approach relies on implicit 3D scene representations. Neural Radiance Fields (NeRF) utilize neural networks for encapsulating the continuous 3D scene structure within the network weights achieving photorealistic synthesis results, however, methods are constrained to per-scene optimization settings which are inefficient in practice. Multi-plane Neural Radiance Fields (MINE) open the door for combining implicit and explicit scene representations. It enables continuous 3D scene representations, especially in the depth dimension, while utilizing the input image features to avoid per-scene optimization. The main drawback of the current literature work in this domain is being constrained to single-view input, limiting the synthesis ability to narrow viewpoint ranges. In this work, we thoroughly examine the performance, generalization, and efficiency of single-view multi-plane neural radiance fields. In addition, we propose a new multiplane NeRF architecture that accepts multiple views to improve the synthesis results and expand the viewing range. Features from the input source frames are effectively fused through a proposed attention-aware fusion module to highlight important information from different viewpoints. Experiments show the effectiveness of attention-based fusion and the promising outcomes of our proposed method when compared to multi-view NeRF and MPI techniques.

Remote Sensing Novel View Synthesis With Implicit Multiplane Representations

SinMPI: Novel View Synthesis from a Single Image with Expanded Multiplane Images

View Synthesis with Multi-scale Cost Aggregation and Confidence Prior

Multi-Plane Neural Radiance Fields for Novel View Synthesis

View synthesis with multiplane images from computationally generated RGB-D light fields

Novel View Synthesis Using Feature-Preserving Depth Map Resampling

Few-Shot Remote Sensing Novel View Synthesis with Geometry Constraint NeRF

CMC: Few-shot Novel View Synthesis via Cross-view Multiplane Consistency

Real-time High-resolution View Synthesis of Complex Scenes with Explicit 3D Visibility Reasoning

View Synthesis with Sculpted Neural Points

Neural 3D Video Synthesis from Multi-view Video

IBRNet: Learning Multi-View Image-Based Rendering

Novel View Synthesis from only a 6-DoF Camera Pose by Two-stage Networks

NeMI: Unifying Neural Radiance Fields with Multiplane Images for Novel View Synthesis

Multi-View Stereo and Depth Priors Guided NeRF for View Synthesis.

High-Fidelity and Real-Time Novel View Synthesis for Dynamic Scenes.

Efficient 3D View Synthesis from Single-Image Utilizing Diffusion Priors

Novel View Synthesis from Single Images via Point Cloud Transformation

Remote Sensing Neural Radiance Fields for Multi-View Satellite Photogrammetry

CompNVS: Novel View Synthesis with Scene Completion

Stereo Magnification: Learning View Synthesis using Multiplane Images