Abstract:The neural radiance fields (NeRF) have advanced the development of 3D volumetric video technology, but the large data volumes they involve pose significant challenges for storage and transmission. To address these problems, the existing solutions typically compress these NeRF representations after the training stage, leading to a separation between representation training and compression. In this paper, we try to directly learn a compact NeRF representation for volumetric video in the training stage based on the proposed rate-aware compression framework. Specifically, for volumetric video, we use a simple yet effective modeling strategy to reduce temporal redundancy for the NeRF representation. Then, during the training phase, an implicit entropy model is utilized to estimate the bitrate of the NeRF representation. This entropy model is then encoded into the bitstream to assist in the decoding of the NeRF representation. This approach enables precise bitrate estimation, thereby leading to a compact NeRF representation. Furthermore, we propose an adaptive quantization strategy and learn the optimal quantization step for the NeRF representations. Finally, the NeRF representation can be optimized by using the rate-distortion trade-off. Our proposed compression framework can be used for different representations and experimental results demonstrate that our approach significantly reduces the storage size with marginal distortion and achieves state-of-the-art rate-distortion performance for volumetric video on the HumanRF and ReRF datasets. Compared to the previous state-of-the-art method TeTriRF, we achieved an approximately -80% BD-rate on the HumanRF dataset and -60% BD-rate on the ReRF dataset.

Static-Scene Video Compression with Neural Radiance Fields

Neural Radiance Field-Assisted Static-Scene Video Coding

NeRFCodec: Neural Feature Compression Meets Neural Radiance Fields for Memory-Efficient Scene Representation

Neural NeRF Compression

JointRF: End-to-End Joint Optimization for Dynamic Neural Radiance Field Representation and Compression

Rate-aware Compression for NeRF-based Volumetric Video

ECRF: Entropy-Constrained Neural Radiance Fields Compression with Frequency Domain Optimization

Efficient Dynamic-NeRF Based Volumetric Video Coding with Rate Distortion Optimization

Neural 3D Scene Compression via Model Compression

NeVRF: Neural Video-based Radiance Fields for Long-duration Sequences

CodecNeRF: Toward Fast Encoding and Decoding, Compact, and High-quality Novel-view Synthesis

HPC: Hierarchical Progressive Coding Framework for Volumetric Video

Fast Dynamic Radiance Fields with Time-Aware Neural Voxels

VideoRF: Rendering Dynamic Radiance Fields as 2D Feature Video Streams

SPC-NeRF: Spatial Predictive Compression for Voxel Based Radiance Field

How Far Can We Compress Instant-NGP-Based NeRF?

Resolution-Agnostic Neural Compression for High-Fidelity Portrait Video Conferencing via Implicit Radiance Fields

Neural Residual Radiance Fields for Streamably Free-Viewpoint Videos

Compressible-composable NeRF via Rank-residual Decomposition

Scene Matters: Model-based Deep Video Compression

TinyNeRF: Towards 100 x Compression of Voxel Radiance Fields