Abstract:We introduce an improved solution to the neural image-based rendering problem in computer vision. Given a set of images taken from a freely moving camera at train time, the proposed approach could synthesize a realistic image of the scene from a novel viewpoint at test time. The key ideas presented in this paper are (i) Recovering accurate camera parameters via a robust pipeline from unposed day-to-day images is equally crucial in neural novel view synthesis problem; (ii) It is rather more practical to model object's content at different resolutions since dramatic camera motion is highly likely in day-to-day unposed images. To incorporate the key ideas, we leverage the fundamentals of scene rigidity, multi-scale neural scene representation, and single-image depth prediction. Concretely, the proposed approach makes the camera parameters as learnable in a neural fields-based modeling framework. By assuming per view depth prediction is given up to scale, we constrain the relative pose between successive frames. From the relative poses, absolute camera pose estimation is modeled via a graph-neural network-based multiple motion averaging within the multi-scale neural-fields network, leading to a single loss function. Optimizing the introduced loss function provides camera intrinsic, extrinsic, and image rendering from unposed images. We demonstrate, with examples, that for a unified framework to accurately model multiscale neural scene representation from day-to-day acquired unposed multi-view images, it is equally essential to have precise camera-pose estimates within the scene representation framework. Without considering robustness measures in the camera pose estimation pipeline, modeling for multi-scale aliasing artifacts can be counterproductive. We present extensive experiments on several benchmark datasets to demonstrate the suitability of our approach.

HashCC: Lightweight Method to Improve the Quality of the Camera-less NeRF Scene Generation

Camera Relocalization in Shadow-free Neural Radiance Fields

Hash Encoding and Brightness Correction in 3D Industrial and Environmental Reconstruction of Tidal Flat Neural Radiation

Dip-NeRF: Depth-Based Anti-Aliased Neural Radiance Fields

Neural Radiance Fields From Sparse RGB-D Images for High-Quality View Synthesis

UP-NeRF: Unconstrained Pose-Prior-Free Neural Radiance Fields

SN 2 eRF: A Framework for Neural Radiance Fields given Sparse and Noisy Poses

Hyb-NeRF: A Multiresolution Hybrid Encoding for Neural Radiance Fields

NeFF: Neural Feature Fields for Few-Shot View Synthesis Addressing the Shape-Radiance Ambiguity

Learning Robust Multi-Scale Representation for Neural Radiance Fields from Unposed Images

HG3-NeRF: Hierarchical Geometric, Semantic, and Photometric Guided Neural Radiance Fields for Sparse View Inputs

LuSh-NeRF: Lighting up and Sharpening NeRFs for Low-light Scenes

Improving Neural Radiance Field using Near-Surface Sampling with Point Cloud Generation

Aleth-NeRF: Low-light Condition View Synthesis with Concealing Fields

Lighting up NeRF via Unsupervised Decomposition and Enhancement

Cross-Ray Neural Radiance Fields for Novel-view Synthesis from Unconstrained Image Collections

Robust Camera Pose Refinement for Multi-Resolution Hash Encoding

MM-NeRF: Large-Scale Scene Representation with Multi-Resolution Hash Grid and Multi-View Priors Features

NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections

Fast High Dynamic Range Radiance Fields for Dynamic Scenes

HollowNeRF: Pruning Hashgrid-Based NeRFs with Trainable Collision Mitigation