Abstract:Neural Radiance Fields (NeRFs) have demonstrated remarkable potential in capturing complex 3D scenes with high fidelity. However, one persistent challenge that hinders the widespread adoption of NeRFs is the computational bottleneck due to the volumetric rendering. On the other hand, 3D Gaussian splatting (3DGS) has recently emerged as an alternative representation that leverages a 3D Gaussisan-based representation and adopts the rasterization pipeline to render the images rather than volumetric rendering, achieving very fast rendering speed and promising image quality. However, a significant drawback arises as 3DGS entails a substantial number of 3D Gaussians to maintain the high fidelity of the rendered images, which requires a large amount of memory and storage. To address this critical issue, we place a specific emphasis on two key objectives: reducing the number of Gaussian points without sacrificing performance and compressing the Gaussian attributes, such as view-dependent color and covariance. To this end, we propose a learnable mask strategy that significantly reduces the number of Gaussians while preserving high performance. In addition, we propose a compact but effective representation of view-dependent color by employing a grid-based neural field rather than relying on spherical harmonics. Finally, we learn codebooks to compactly represent the geometric attributes of Gaussian by vector quantization. With model compression techniques such as quantization and entropy coding, we consistently show over 25$\times$ reduced storage and enhanced rendering speed, while maintaining the quality of the scene representation, compared to 3DGS. Our work provides a comprehensive framework for 3D scene representation, achieving high performance, fast training, compactness, and real-time rendering. Our project page is available at <a class="link-external link-https" href="https://maincold2.github.io/c3dgs/" rel="external noopener nofollow">this https URL</a>.

ChromaDistill: Colorizing Monochrome Radiance Fields with Knowledge Distillation

MPS-NeRF: Generalizable 3D Human Rendering from Multiview Images

Colorizing Monochromatic Radiance Fields

Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fields

The Potential of Diffusion-Based Near-Infrared Image Colorization

Neural Radiance Fields From Sparse RGB-D Images for High-Quality View Synthesis

ID-NeRF: Indirect Diffusion-guided Neural Radiance Fields for Generalizable View Synthesis

SNeRF: Stylized Neural Implicit Representations for 3D Scenes

Beyond RGB: Scene-Property Synthesis with Neural Radiance Fields

Novel View Synthesis and Dataset Augmentation for Hyperspectral Data Using NeRF

DNGaussian: Optimizing Sparse-View 3D Gaussian Radiance Fields with Global-Local Depth Normalization

DistillNeRF: Perceiving 3D Scenes from Single-Glance Images by Distilling Neural Fields and Foundation Model Features

Diffusing Colors: Image Colorization with Text Guided Diffusion

Denoising Diffusion via Image-Based Rendering

HyperColor: A HyperNetwork Approach for Synthesizing Auto-colored 3D Models for Game Scenes Population

Neural Colour Correction for Indoor 3D Reconstruction Using RGB-D Data

NerfDiff: Single-image View Synthesis with NeRF-guided Distillation from 3D-aware Diffusion

Neural Radiance Flow for 4D View Synthesis and Video Processing

Novel View Synthesis from a Single RGBD Image for Indoor Scenes

ReconFusion: 3D Reconstruction with Diffusion Priors

Compact 3D Gaussian Representation for Radiance Field