Abstract:Neural rendering bakes global illumination and other computationally costly effects into the weights of a neural network, allowing to efficiently synthesize photorealistic images without relying on path tracing. In neural rendering approaches, G‐buffers obtained from rasterization through direct rendering provide information regarding the scene such as position, normal, and textures to the neural network, achieving accurate and stable rendering quality in real‐time. However, due to the use of G‐buffers, existing methods struggle to accurately render transparency and refraction effects, as G‐buffers do not capture any ray information from multiple light ray bounces. This limitation results in blurriness, distortions, and loss of detail in rendered images that contain transparency and refraction, and is particularly notable in scenes with refracted objects that have high‐frequency textures. In this work, we propose a neural network architecture to encode critical rendering information, including texture coordinates from refracted rays, and enable reconstruction of high‐frequency textures in areas with refraction. Our approach is able to achieve accurate refraction rendering in challenging scenes with a diversity of overlapping transparent objects. Experimental results demonstrate that our method can interactively render high quality refraction effects with global illumination, unlike existing neural rendering approaches. Our code can be found at https://github.com/ziyangz5/CrystalNet

Refractive Index and Transmission Coefficient Recovery Algorithm of Crystal-like Objects

Material Recovery Algorithm of Translucent Object under Complex Environment Illumination

Single Image Reflection Removal with Absorption Effect

A New Iterative Algorithm to Reconstruct the Refractive Index

CrystalNet: Texture‐Aware Neural Refraction Baking for Global Illumination

Estimation of Refractive Index for Biological Tissue Using Micro-Optical Coherence Tomography

A Novel Description of Refractive Index in Irregular Inhomogeneous Optical Medium

Interactive Rendering of Non‐Constant, Refractive Media Using the Ray Equations of Gradient‐Index Optics

Reconstruction of Optical Properties in Turbid Media: Omitting the Need of the Collimated Transmission for an Integrating Sphere Setup

Linear Partial Derivative Matrix for Iterative Algorithm to Reconstruct Refractive Index from Refraction Angle Data

Numerical reconstruction of turbid slab optical properties using global optimization algorithms

Inverse Method to Estimate Temperature-Dependent Thermal and Optical Properties of Semitransparent Media

Theoretical Study of Light Refraction in Three-Dimensional Photonic Crystals

Reflectance Reconstruction of Objects with Spatially Varying Materials

Real-time Rendering of Refracting Transmissive Objects with Multi-Scale Rough Surfaces

Transparent Object Reconstruction via Coded Transport of Intensity

Real-time rendering of rough refraction

An Estimation For Complex Index Of Refraction Based On Active Polarized Reflection Measurement

Inverse Diffusion Approximation for Extraction of Scattering and Absorption Coefficients in Highly Scattering Media

Modeling and Rendering of Heterogeneous Translucent Materials Using the Diffusion Equation

A practical algorithm for rendering interreflections with all-frequency BRDFs