Abstract:Neural Radiance Fields (NeRFs) have achieved great success in the past few years. However, most current methods still require intensive resources due to ray marching-based rendering. To construct urban-level radiance fields efficiently, we design Deformable Neural Mesh Primitive~(DNMP), and propose to parameterize the entire scene with such primitives. The DNMP is a flexible and compact neural variant of classic mesh representation, which enjoys both the efficiency of rasterization-based rendering and the powerful neural representation capability for photo-realistic image synthesis. Specifically, a DNMP consists of a set of connected deformable mesh vertices with paired vertex features to parameterize the geometry and radiance information of a local area. To constrain the degree of freedom for optimization and lower the storage budgets, we enforce the shape of each primitive to be decoded from a relatively low-dimensional latent space. The rendering colors are decoded from the vertex features (interpolated with rasterization) by a view-dependent MLP. The DNMP provides a new paradigm for urban-level scene representation with appealing properties: $(1)$ High-quality rendering. Our method achieves leading performance for novel view synthesis in urban scenarios. $(2)$ Low computational costs. Our representation enables fast rendering (2.07ms/1k pixels) and low peak memory usage (110MB/1k pixels). We also present a lightweight version that can run 33$\times$ faster than vanilla NeRFs, and comparable to the highly-optimized Instant-NGP (0.61 vs 0.71ms/1k pixels). Project page: \href{<a class="link-external link-https" href="https://dnmp.github.io/" rel="external noopener nofollow">this https URL</a>}{<a class="link-external link-https" href="https://dnmp.github.io/" rel="external noopener nofollow">this https URL</a>}.

PARF: Primitive-Aware Radiance Fusion for Indoor Scene Novel View Synthesis

PARF: Primitive-Aware Radiance Fusion for Indoor Scene Novel View Synthesis

DisCoScene: Spatially Disentangled Generative Radiance Fields for Controllable 3D-Aware Scene Synthesis

I2-SDF: Intrinsic Indoor Scene Reconstruction and Editing Via Raytracing in Neural SDFs

Learning Object-Compositional Neural Radiance Field for Editable Scene Rendering

Progressively Optimized Local Radiance Fields for Robust View Synthesis

ERF: Explicit Radiance Field Reconstruction From Scratch

PVSeRF: Joint Pixel-, Voxel- and Surface-Aligned Radiance Field for Single-Image Novel View Synthesis

An Exploration of Neural Radiance Field Scene Reconstruction: Synthetic, Real-world and Dynamic Scenes

Improved Neural Radiance Fields Using Pseudo-depth and Fusion

Urban Radiance Field Representation with Deformable Neural Mesh Primitives

CeRF: Convolutional Neural Radiance Fields for New View Synthesis with Derivatives of Ray Modeling

Urban Radiance Fields

Neural Relightable Participating Media Rendering

Structured-NeRF: Hierarchical Scene Graph with Neural Representation

Efficient Neural Radiance Fields with Learned Depth-Guided Sampling

NeRF-SDP: Efficient Generalizable Neural Radiance Field with Scene Depth Perception.

Holistic Inverse Rendering of Complex Facade via Aerial 3D Scanning

3D Reconstruction and New View Synthesis of Indoor Environments based on a Dual Neural Radiance Field

A New Era of Indoor Scene Reconstruction: A Survey

Neural Fields meet Explicit Geometric Representation for Inverse Rendering of Urban Scenes