Abstract:The generation of global illumination in real time has been a long-standing challenge in the graphics community, particularly in dynamic scenes with complex illumination. Recent neural rendering techniques have shown great promise by utilizing neural networks to represent the illumination of scenes and then decoding the final radiance. However, incorporating object parameters into the representation may limit their effectiveness in handling fully dynamic scenes. This work presents a neural rendering approach, dubbed LightFormer , that can generate realistic global illumination for fully dynamic scenes, including dynamic lighting, materials, cameras, and animated objects, in real time. Inspired by classic many-lights methods, the proposed approach focuses on the neural representation of light sources in the scene rather than the entire scene, leading to the overall better generalizability. The neural prediction is achieved by leveraging the virtual point lights and shading clues for each light. Specifically, two stages are explored. In the light encoding stage, each light generates a set of virtual point lights in the scene, which are then encoded into an implicit neural light representation, along with screen-space shading clues like visibility. In the light gathering stage, a pixel-light attention mechanism composites all light representations for each shading point. Given the geometry and material representation, in tandem with the composed light representations of all lights, a lightweight neural network predicts the final radiance. Experimental results demonstrate that the proposed LightFormer can yield reasonable and realistic global illumination in fully dynamic scenes with real-time performance.

Direct Lighting under Dynamic Local Area Light Sources.

Virtual Area Light for Dynamic and Spatially-Varying Area Lighting

Direct illumination for many area lights.

Real-Time Rendering under Dynamic Environment Lighting

Real-time rendering with complex natural illumination

A Framework for Lighting Detection and Scene Rendering in AR Environment

A Virtual Point Light Generation Method in Close-Range Area

A real-time global relighting algorithm

Research on Real-time Rendering under Complex Area Lighting

Joint Optimization of Triangle Mesh, Material, and Light from Neural Fields with Neural Radiance Cache

Accelerated Radiosity Method for Complex Environments

Light Field Rendering of Dynamic Scene

Volume-based indirect lighting with irradiance decomposition.

A Survey on Real-Time Rendering Algorithms for Dynamic Scenes under Complex Environment Lighting

Structure-Aware Lighting Design for Volume Visualization.

Real-Time Illumination of Complex Lights Using Divided Sampling

Efficient Real-Time Dynamic Diffuse Global Illumination Using Signed Distance Fields

Rendering Dynamic Scene based on Ambient Light Field and Diffuse Light Field

Signed Distance Fields Dynamic Diffuse Global Illumination

LightFormer: Light-Oriented Global Neural Rendering in Dynamic Scene

Dynamic Voxel‐Based Global Illumination