How Far Can We Compress Instant-NGP-Based NeRF?

Yihang Chen,Qianyi Wu,Mehrtash Harandi,Jianfei Cai
2024-06-06
Abstract:In recent years, Neural Radiance Field (NeRF) has demonstrated remarkable capabilities in representing 3D scenes. To expedite the rendering process, learnable explicit representations have been introduced for combination with implicit NeRF representation, which however results in a large storage space requirement. In this paper, we introduce the Context-based NeRF Compression (CNC) framework, which leverages highly efficient context models to provide a storage-friendly NeRF representation. Specifically, we excavate both level-wise and dimension-wise context dependencies to enable probability prediction for information entropy reduction. Additionally, we exploit hash collision and occupancy grids as strong prior knowledge for better context modeling. To the best of our knowledge, we are the first to construct and exploit context models for NeRF compression. We achieve a size reduction of 100$\times$ and 70$\times$ with improved fidelity against the baseline Instant-NGP on Synthesic-NeRF and Tanks and Temples datasets, respectively. Additionally, we attain 86.7\% and 82.3\% storage size reduction against the SOTA NeRF compression method BiRF. Our code is available here: <a class="link-external link-https" href="https://github.com/YihangChen-ee/CNC" rel="external noopener nofollow">this https URL</a>.
Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
The paper aims to address the issue of storage space requirements for Neural Radiance Fields (NeRF) models, particularly those using hybrid representations such as Instant-NGP (INGP). While INGP and similar methods have improved the rendering speed of NeRF models, they require a large amount of storage space, especially for the feature embeddings. The authors propose a Context-based NeRF Compression (CNC) framework that leverages efficient context models to reduce the storage size of feature embeddings while maintaining high fidelity and rendering speed. The key contributions of the paper are: 1. **Introduction of Context Models for NeRF Compression**: The authors are the first to propose modeling the contexts of INGP's multi-resolution hashing feature embeddings to reduce storage size while maintaining fidelity and speed. 2. **Customized Context Models**: They design specialized context models that effectively build multi-level and cross-dimensional dependencies for INGP hash embeddings. They also utilize hash collision and occupancy grid as strong prior knowledge to provide more accurate contexts. 3. **Significant Size Reduction and Improved Fidelity**: Extensive experiments demonstrate that the CNC framework achieves a size reduction of over 100x and 70x on the Synthetic-NeRF and Tanks and Temples datasets, respectively, while simultaneously improving fidelity. The approach outperforms the state-of-the-art NeRF compression algorithm, BiRF, with over 80% size reduction.