Abstract:We present GI-GS, a novel inverse rendering framework that leverages 3D Gaussian Splatting (3DGS) and deferred shading to achieve photo-realistic novel view synthesis and relighting. In inverse rendering, accurately modeling the shading processes of objects is essential for achieving high-fidelity results. Therefore, it is critical to incorporate global illumination to account for indirect lighting that reaches an object after multiple bounces across the scene. Previous 3DGS-based methods have attempted to model indirect lighting by characterizing indirect illumination as learnable lighting volumes or additional attributes of each Gaussian, while using baked occlusion to represent shadow effects. These methods, however, fail to accurately model the complex physical interactions between light and objects, making it impossible to construct realistic indirect illumination during relighting. To address this limitation, we propose to calculate indirect lighting using efficient path tracing with deferred shading. In our framework, we first render a G-buffer to capture the detailed geometry and material properties of the scene. Then, we perform physically-based rendering (PBR) only for direct lighting. With the G-buffer and previous rendering results, the indirect lighting can be calculated through a lightweight path tracing. Our method effectively models indirect lighting under any given lighting conditions, thereby achieving better novel view synthesis and relighting. Quantitative and qualitative results show that our GI-GS outperforms existing baselines in both rendering quality and efficiency.

Practical and Accurate Reconstruction of an Illuminant's Spectral Power Distribution for Inverse Rendering Pipelines

Non-Line-of-Sight Reconstruction using Efficient Transient Rendering

Deep Inverse Rendering for Practical Object Appearance Scan with Uncalibrated Illumination

A New Radiosity Approach by Procedural Refinements for Realistic Image Sythesis

IntrinsicAnything: Learning Diffusion Priors for Inverse Rendering Under Unknown Illumination

SIRe-IR: Inverse Rendering for BRDF Reconstruction with Shadow and Illumination Removal in High-Illuminance Scenes

GUS-IR: Gaussian Splatting with Unified Shading for Inverse Rendering

Eclipse: Disambiguating Illumination and Materials using Unintended Shadows

Fast Spectral Reflectance Recovery Using DLP Projector

IRIS: Inverse Rendering of Indoor Scenes from Low Dynamic Range Images

Physics-based Indirect Illumination for Inverse Rendering

GI-GS: Global Illumination Decomposition on Gaussian Splatting for Inverse Rendering

Spectral Reconstruction from RGB Imagery: A Potential Option for Infinite Spectral Data?

Modeling Indirect Illumination for Inverse Rendering

Reconstruction of spectral irradiance in a real application with a multi-channel spectral sensor using convolutional neural networks.

IRGS: Inter-Reflective Gaussian Splatting with 2D Gaussian Ray Tracing

GS-ID: Illumination Decomposition on Gaussian Splatting via Diffusion Prior and Parametric Light Source Optimization

Spectral Reconstruction from Dispersive Blur: A Novel Light Efficient Spectral Imager.

IRCasTRF: Inverse Rendering by Optimizing Cascaded Tensorial Radiance Fields, Lighting, and Materials From Multi-view Images

Spectral Reflectance Reconstruction Based on Spectral Optimization of Light Source

RISE-SDF: a Relightable Information-Shared Signed Distance Field for Glossy Object Inverse Rendering