Abstract:Since the introduction of NeRFs, considerable attention has been focused on improving their training and inference times, leading to the development of Fast-NeRFs models. Despite demonstrating impressive rendering speed and quality, the rapid convergence of such models poses challenges for further improving reconstruction quality. Common strategies to improve rendering quality involves augmenting model parameters or increasing the number of sampled points. However, these computationally intensive approaches encounter limitations in achieving significant quality enhancements. This study introduces a model-agnostic framework inspired by Sparsely-Gated Mixture of Experts to enhance rendering quality without escalating computational complexity. Our approach enables specialization in rendering different scene components by employing a mixture of experts with varying resolutions. We present a novel gate formulation designed to maximize expert capabilities and propose a resolution-based routing technique to effectively induce sparsity and decompose scenes. Our work significantly improves reconstruction quality while maintaining competitive performance.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: to improve the rendering quality of Neural Radiance Fields (NeRFs) models without significantly increasing the computational complexity. Specifically, although the Fast - NeRFs model performs well in terms of rendering speed and quality, its rapid convergence poses a challenge to further improving the reconstruction quality. Common improvement strategies, such as increasing the number of model parameters or sampling points, can improve the quality to a certain extent, but will lead to a significant increase in computational cost. To solve these problems, the author proposes a method based on the Sparse Mixture of Experts (MoE) framework, which can significantly improve the rendering quality without increasing the computational complexity. This framework achieves this goal by allowing expert models of different resolutions to focus on different parts of the rendering scene. The following are the main contributions of this method: 1. **Proposing a general framework**: Based on sparse MoE models of different resolutions, it can significantly improve the rendering quality while maintaining competitive training and inference times. 2. **A new gating mechanism**: Inspired by the Fast - NeRF model, a new gating formula is designed to maximize the ability of the mixture of experts. 3. **Resolution - based routing technology**: A new routing technology is introduced, which gives priority to assigning tokens to low - resolution models and minimizes the use of high - resolution models, thereby decomposing the scene on a frequency basis. 4. **Extensive experimental verification**: Through experiments with different NeRF architectures and datasets, higher rendering quality and efficiency are demonstrated. This method can not only effectively improve the rendering quality, but also remain efficient in terms of computational resources, solving the trade - off problem between quality improvement and computational cost in existing methods.

Boost Your NeRF: A Model-Agnostic Mixture of Experts Framework for High Quality and Efficient Rendering

Adaptive Multi-NeRF: Exploit Efficient Parallelism in Adaptive Multiple Scale Neural Radiance Field Rendering

RoGUENeRF: A Robust Geometry-Consistent Universal Enhancer for NeRF

Reconstructive Latent-Space Neural Radiance Fields for Efficient 3D Scene Representations

MixRT: Mixed Neural Representations For Real-Time NeRF Rendering

InterNeRF: Scaling Radiance Fields via Parameter Interpolation

Towards a Robust Framework for NeRF Evaluation

Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields

PyNeRF: Pyramidal Neural Radiance Fields

From NeRFLiX to NeRFLiX++: A General NeRF-Agnostic Restorer Paradigm

Aug-NeRF: Training Stronger Neural Radiance Fields with Triple-Level Physically-Grounded Augmentations

GANeRF: Leveraging Discriminators to Optimize Neural Radiance Fields

SGCNeRF: Few-Shot Neural Rendering via Sparse Geometric Consistency Guidance

Enhance-NeRF: Multiple Performance Evaluation for Neural Radiance Fields

GUMBEL-NERF: Representing Unseen Objects as Part-Compositional Neural Radiance Fields

A General Implicit Framework for Fast NeRF Composition and Rendering

MD-NeRF: Enhancing Large-Scale Scene Rendering and Synthesis With Hybrid Point Sampling and Adaptive Scene Decomposition

BoostMVSNeRFs: Boosting MVS-based NeRFs to Generalizable View Synthesis in Large-scale Scenes

Prior-Driven NeRF: Prior Guided Rendering

Blending Distributed NeRFs with Tri-stage Robust Pose Optimization

Re-Nerfing: Improving Novel View Synthesis through Novel View Synthesis