Abstract:The neural radiance fields (NeRF) have advanced the development of 3D volumetric video technology, but the large data volumes they involve pose significant challenges for storage and transmission. To address these problems, the existing solutions typically compress these NeRF representations after the training stage, leading to a separation between representation training and compression. In this paper, we try to directly learn a compact NeRF representation for volumetric video in the training stage based on the proposed rate-aware compression framework. Specifically, for volumetric video, we use a simple yet effective modeling strategy to reduce temporal redundancy for the NeRF representation. Then, during the training phase, an implicit entropy model is utilized to estimate the bitrate of the NeRF representation. This entropy model is then encoded into the bitstream to assist in the decoding of the NeRF representation. This approach enables precise bitrate estimation, thereby leading to a compact NeRF representation. Furthermore, we propose an adaptive quantization strategy and learn the optimal quantization step for the NeRF representations. Finally, the NeRF representation can be optimized by using the rate-distortion trade-off. Our proposed compression framework can be used for different representations and experimental results demonstrate that our approach significantly reduces the storage size with marginal distortion and achieves state-of-the-art rate-distortion performance for volumetric video on the HumanRF and ReRF datasets. Compared to the previous state-of-the-art method TeTriRF, we achieved an approximately -80% BD-rate on the HumanRF dataset and -60% BD-rate on the ReRF dataset.

Compression-Aware Tuning for Compressing Volumetric Radiance Fields

TinyNeRF: Towards 100 x Compression of Voxel Radiance Fields

Joint Optimization of Triangle Mesh, Material, and Light from Neural Fields with Neural Radiance Cache

JointRF: End-to-End Joint Optimization for Dynamic Neural Radiance Field Representation and Compression

SPC-NeRF: Spatial Predictive Compression for Voxel Based Radiance Field

Distilled Low Rank Neural Radiance Field with Quantization for Light Field Compression

CVT-xRF: Contrastive In-Voxel Transformer for 3D Consistent Radiance Fields from Sparse Inputs

Compact Real-time Radiance Fields with Neural Codebook

Content-Aware Radiance Fields: Aligning Model Complexity with Scene Intricacy Through Learned Bitwidth Quantization

Neural NeRF Compression

ECRF: Entropy-Constrained Neural Radiance Fields Compression with Frequency Domain Optimization

Rate-aware Compression for NeRF-based Volumetric Video

Improved Neural Radiance Fields Using Pseudo-depth and Fusion

NeRFCodec: Neural Feature Compression Meets Neural Radiance Fields for Memory-Efficient Scene Representation

VGOS: Voxel Grid Optimization for View Synthesis from Sparse Inputs

VQ-NeRF: Vector Quantization Enhances Implicit Neural Representations

Fast Dynamic Radiance Fields with Time-Aware Neural Voxels

Accelerating Neural Radiance Field by network pruning

CeRF: Convolutional Neural Radiance Fields for New View Synthesis with Derivatives of Ray Modeling

Compressible-composable NeRF via Rank-residual Decomposition

Expansive Supervision for Neural Radiance Field