NeRF-XL: Scaling NeRFs with Multiple GPUs

Ruilong Li,Sanja Fidler,Angjoo Kanazawa,Francis Williams
2024-04-25
Abstract:We present NeRF-XL, a principled method for distributing Neural Radiance Fields (NeRFs) across multiple GPUs, thus enabling the training and rendering of NeRFs with an arbitrarily large capacity. We begin by revisiting existing multi-GPU approaches, which decompose large scenes into multiple independently trained NeRFs, and identify several fundamental issues with these methods that hinder improvements in reconstruction quality as additional computational resources (GPUs) are used in training. NeRF-XL remedies these issues and enables the training and rendering of NeRFs with an arbitrary number of parameters by simply using more hardware. At the core of our method lies a novel distributed training and rendering formulation, which is mathematically equivalent to the classic single-GPU case and minimizes communication between GPUs. By unlocking NeRFs with arbitrarily large parameter counts, our approach is the first to reveal multi-GPU scaling laws for NeRFs, showing improvements in reconstruction quality with larger parameter counts and speed improvements with more GPUs. We demonstrate the effectiveness of NeRF-XL on a wide variety of datasets, including the largest open-source dataset to date, MatrixCity, containing 258K images covering a 25km^2 city area.
Computer Vision and Pattern Recognition,Distributed, Parallel, and Cluster Computing,Graphics
What problem does this paper attempt to address?
This paper addresses the problem of training and rendering neural radiance fields (NeRFs) using multiple GPUs, which fail to improve the reconstruction quality effectively when adding more GPUs. NeRF-XL proposes a new distributed training and rendering formulation that reduces communication between GPUs and allows for the addition of more hardware to scale the number of parameters in NeRF, thus achieving improvements in quality and speed. The study also reveals the scaling behavior of NeRF with multiple GPUs.