Abstract:Real‐time global illumination is a highly desirable yet challenging task in computer graphics. Existing works well solving this problem are mostly based on some kind of precomputed data (caches), while the final results depend significantly on the quality of the caches. In this paper, we propose a learning‐based pipeline that can reproduce a wide range of complex light transport phenomena, including high‐frequency glossy interreflection, at any viewpoint in real time (> 90 frames per‐second), using information from imperfect caches stored at the barycentre of every triangle in a 3D scene. These caches are generated at a precomputation stage by a physically‐based offline renderer at a low sampling rate (e.g., 32 samples per‐pixel) and a low image resolution (e.g., 64×16). At runtime, a deep radiance reconstruction method based on a dedicated neural network is then involved to reconstruct a high‐quality radiance map of full global illumination at any viewpoint from these imperfect caches, without introducing noise and aliasing artifacts. To further improve the reconstruction accuracy, a new feature fusion strategy is designed in the network to better exploit useful contents from cheap G‐buffers generated at runtime. The proposed framework ensures high‐quality rendering of images for moderate‐sized scenes with full global illumination effects, at the cost of reasonable precomputation time. We demonstrate the effectiveness and efficiency of the proposed pipeline by comparing it with alternative strategies, including real‐time path tracing and precomputed radiance transfer.

Application of Clustering for Augmented Reality by Instant Radiosity

Efficient screen space approach for high complex dynamic scenes in real-time rendering

Cloud-based Fusion Rendering for AR Applications

A Framework for Lighting Detection and Scene Rendering in AR Environment

Real time stereo rendering for augmented reality on 3DTV system

Cloud-Based Dynamic GI for Shared VR Experiences

Efficient clustering and visibility calculation for global illumination

Accelerated Radiosity Method for Complex Environments

Parallel Shooting and Bouncing Ray Method on Gpu Clusters for Analysis of Electromagnetic Scattering

Relighting in spatial augmented reality

Immersive Virtual Tour Using Multi-projection with Remote Cluster Rendering

Survey of instant radiosity

Radiance Transfer Biclustering for Real-Time All-Frequency Biscale Rendering

Real-Time Lighting Estimation for Augmented Reality Via Differentiable Screen-Space Rendering

A Real-time Method for Inserting Virtual Objects into Neural Radiance Fields

Real-time rendering with complex natural illumination

Parallel Rendering Based on Optical-mapping Virtual Objects

Rendering With Spherical Radiance Transport Maps

Real‐time Deep Radiance Reconstruction from Imperfect Caches

An Information-Centric Architecture for Server Clustering Towards 3D Data-Intensive Applications.

Instant-3D: Instant Neural Radiance Field Training Towards On-Device AR/VR 3D Reconstruction