Abstract:Geographical real-scene 3D scenes are an important national digital infrastructure,which extends geographic information from 2D to 3D.Real-scene 3D data are stored and expressed in the form of an explicit 3D model,which has the problems of large amount of data and rough visualization effect.Neural Radiance Field(NeRF),realizing differentiable rendering based on neural implicit volume representation,is an innovative approach of high-quality view synthesis.First proposed by Mildenhall et al.(2020),NeRF has become one of the hottest research direction in the field of computer vision due to its realistic view synthesis effect.A large amount of literature about NeRF have been published since NeRF was proposed,and the application of NeRF in large-scale real-scene 3D visualization has begun to attract the attention of some published papers. View synthesis,which uses sparse 2D images to generate realistic new views at any viewpoint in 3D space without the reconstruction of 3D models,is a novel way to realize the representation of 3D scenes.The development of view synthesis technology has gone through several stages:image mosaicking,3D model reprojection,view interpolation,and volume representing technology.NeRF,as an innovative approach of view synthesis,samples 5D coordinates(location and viewing direction)along camera rays,feeds those locations into a multilayer perceptron network to produce color and volume density,and uses volume rendering techniques to composite these values into a new image.NeRF not only produces remarkably higher-quality rendering than prior volumetric approaches but also requires just a fraction of the storage cost of other sampled volumetric representations.However,it faces problems such as requirements for high quality of source data,failure to support dynamic objects,low efficiency in processing,and single type of render target.Moreover,NeRF-related research are mostly conducted based on laboratory environment or standardized data at present.Due to these drawbacks,many obstacles need to be overcome before applying NeRF to large-scale real-scene 3D visualization. This paper reviews the workaround of unbounded scene,aliasing,luminosity consistency,scene relighting,and visibility field in the Block-NeRF algorithm,a variant of NeRF that can represent large-scale environments(Tancik et al.2022).The Block-NeRF algorithm splits the environment into a set of Block-NeRFs that can be independently trained in parallel and composited during inference,and it selects relevant Block-NeRFs for rendering,which are then composited smoothly when traversing the scene.To aid with this compositing,Block-NeRF optimizes the appearance codes to match lighting conditions.It trains individual Block-NeRFs using techniques such as appearance embeddings,learned pose refinement,exposure input,transient objects,and visibility prediction. Nowadays,the studies of applying NeRF to large-scale real-scene 3D visualization are being conducted extensively,which has attracted much attention and plays a pioneering,leading role in further research.The results of these studies achieve the most basic 3D visualization large-scale real-scene effect,but they are limited by some conditions and their universality needs to be strongly improved.Because NeRF is still far from practical applications in producing large-scale real-scene 3D visualization,any slight progressive exploration is likely to become a continuous research hotspot.This paper identifies the challenges of NeRF research,including multisource data fusion,visual effect optimization,and virtual environment perception,which need more research.

NeRF synthesis with shading guidance

Neural Radiance Fields From Sparse RGB-D Images for High-Quality View Synthesis

NeRF-Texture: Synthesizing Neural Radiance Field Textures

Improving Neural Radiance Fields with Depth-aware Optimization for Novel View Synthesis

NeRF-SDP: Efficient Generalizable Neural Radiance Field with Scene Depth Perception.

Collaborative neural radiance fields for novel view synthesis

StructNeRF: Neural Radiance Fields for Indoor Scenes With Structural Hints

NeRFReN: Neural Radiance Fields with Reflections

NeRFusion: Fusing Radiance Fields for Large-Scale Scene Reconstruction

Progress in Neural Radiance Field and Its Application in Large-Scale Real-Scene 3D Visualization

SN 2 eRF: A Framework for Neural Radiance Fields given Sparse and Noisy Poses

NeFF: Neural Feature Fields for Few-Shot View Synthesis Addressing the Shape-Radiance Ambiguity

NeRF: Neural Radiance Field in 3D Vision, A Comprehensive Review

NeRF On-the-go: Exploiting Uncertainty for Distractor-free NeRFs in the Wild

Shadow-Aware Point-Based Neural Radiance Fields for High-Resolution Remote Sensing Novel View Synthesis

HG3-NeRF: Hierarchical Geometric, Semantic, and Photometric Guided Neural Radiance Fields for Sparse View Inputs

IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesis

RDNeRF: Relative Depth Guided NeRF for Dense Free View Synthesis

MD-NeRF: Enhancing Large-Scale Scene Rendering and Synthesis With Hybrid Point Sampling and Adaptive Scene Decomposition

Beyond RGB: Scene-Property Synthesis with Neural Radiance Fields

An Exploration of Neural Radiance Field Scene Reconstruction: Synthetic, Real-world and Dynamic Scenes