Abstract:Neural Radiance Fields (NeRFs) have achieved great success in the past few years. However, most current methods still require intensive resources due to ray marching-based rendering. To construct urban-level radiance fields efficiently, we design Deformable Neural Mesh Primitive~(DNMP), and propose to parameterize the entire scene with such primitives. The DNMP is a flexible and compact neural variant of classic mesh representation, which enjoys both the efficiency of rasterization-based rendering and the powerful neural representation capability for photo-realistic image synthesis. Specifically, a DNMP consists of a set of connected deformable mesh vertices with paired vertex features to parameterize the geometry and radiance information of a local area. To constrain the degree of freedom for optimization and lower the storage budgets, we enforce the shape of each primitive to be decoded from a relatively low-dimensional latent space. The rendering colors are decoded from the vertex features (interpolated with rasterization) by a view-dependent MLP. The DNMP provides a new paradigm for urban-level scene representation with appealing properties: $(1)$ High-quality rendering. Our method achieves leading performance for novel view synthesis in urban scenarios. $(2)$ Low computational costs. Our representation enables fast rendering (2.07ms/1k pixels) and low peak memory usage (110MB/1k pixels). We also present a lightweight version that can run 33$\times$ faster than vanilla NeRFs, and comparable to the highly-optimized Instant-NGP (0.61 vs 0.71ms/1k pixels). Project page: \href{<a class="link-external link-https" href="https://dnmp.github.io/" rel="external noopener nofollow">this https URL</a>}{<a class="link-external link-https" href="https://dnmp.github.io/" rel="external noopener nofollow">this https URL</a>}.

NeAI: A Pre-convoluted Representation for Plug-and-Play Neural Ambient Illumination

A Pre-convolved Representation for Plug-and-Play Neural Illumination Fields

ReN Human: Learning Relightable Neural Implicit Surfaces for Animatable Human Rendering

Aleth-NeRF: Illumination Adaptive NeRF with Concealing Field Assumption

Neural Global Illumination Via Superposed Deformable Feature Fields

IllumiNeRF: 3D Relighting without Inverse Rendering

Neural Fields meet Explicit Geometric Representation for Inverse Rendering of Urban Scenes

Neural Fields Meet Explicit Geometric Representations for Inverse Rendering of Urban Scenes

IBL-NeRF: Image-Based Lighting Formulation of Neural Radiance Fields

Aleth-NeRF: Low-light Condition View Synthesis with Concealing Fields

NeILF: Neural Incident Light Field for Physically-based Material Estimation

NeRFactor

Ambient-NeRF: light train enhancing neural radiance fields in low-light conditions with ambient-illumination

IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesis

NeRF as Non-Distant Environment Emitter in Physics-based Inverse Rendering

Neural Relightable Participating Media Rendering

Object-Based Illumination Estimation with Rendering-Aware Neural Networks

Reconstructive Latent-Space Neural Radiance Fields for Efficient 3D Scene Representations

Enhancing Neural Radiance Fields with Depth and Normal Completion Priors from Sparse Views

Urban Radiance Field Representation with Deformable Neural Mesh Primitives

ABLE-NeRF: Attention-Based Rendering with Learnable Embeddings for Neural Radiance Field