Abstract:Achieving high efficiency in modern photorealistic rendering hinges on using Monte Carlo sampling distributions that closely approximate the illumination integral estimated for every pixel. Samples are typically generated from a set of simple distributions, each targeting a different factor in the integrand, which are combined via multiple importance sampling. The resulting mixture distribution can be far from the actual product of all factors, leading to sub-optimal variance even for direct-illumination estimation. We present a learning-based method that uses normalizing flows to efficiently importance sample illumination product integrals, e.g., the product of environment lighting and material terms. Our sampler composes a flow head warp with an emitter tail warp. The small conditional head warp is represented by a neural spline flow, while the large unconditional tail is discretized per environment map and its evaluation is instant. If the conditioning is low-dimensional, the head warp can be also discretized to achieve even better performance. We demonstrate variance reduction over prior methods on a range of applications comprising complex geometry, materials and illumination.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper aims to solve the problems of large variance and low efficiency encountered when the Monte Carlo integration method is used to estimate the illumination integral distribution in modern photo - realistic rendering. Specifically, the paper proposes a learning method based on normalizing flows. By combining the neural spline flow and the emitter tail warp, it realizes the approximate product importance sampling of ambient illumination and other factors (such as the cosine term and BRDF). #### Main problem description 1. **Limitations of existing methods**: - Existing multiple importance sampling (MIS) methods usually combine multiple simple sampling distributions, and each distribution is for different factors in the integral formula. However, these mixed distributions are often far from the actual product of factors, resulting in sub - optimal variance even in direct illumination estimation. - In complex material and illumination configurations, this gap is particularly obvious, affecting the rendering efficiency. 2. **Objectives**: - Propose a new method that can more accurately approximate the product distribution of the illumination integral, thereby significantly reducing the variance and improving the rendering quality. - The method should be able to efficiently handle complex geometric structures, materials and illumination conditions. #### Solution overview The method proposed in the paper utilizes normalizing flows (NFs), especially the neural spline flow, to generate samples so that their distribution is close to the illumination integral distribution. The specific steps are as follows: - **Head warp**: Use the neural spline flow to deform the uniform distribution into an intermediate distribution, which is then further transformed into the target product probability density function (PDF). - **Tail warp**: Pre - compute the unconditional tail warp of each environment map to quickly evaluate the impact of complex ambient illumination. - **Combined transformation**: By combining the head and tail warps, the model can effectively generate samples close to the real product distribution. #### Key contributions 1. **Novel combination method**: Propose a combination method based on normalizing flows, which combines a small general - purpose head warp with a large pre - computed tail warp to achieve efficient product importance sampling. 2. **New neural architecture**: Introduce a new neural architecture based on circular rational quadratic splines. 3. **Integration in practical applications**: Demonstrate the performance improvement of this method in multiple application scenarios and successfully integrate it into existing rendering systems, such as the Mitsuba 3 renderer. Through these improvements, the paper solves the problems of large variance and low efficiency in traditional multiple importance sampling methods and provides a more effective solution for photo - realistic rendering.

Neural Product Importance Sampling via Warp Composition

Online Neural Path Guiding with Normalized Anisotropic Spherical Gaussians

Neural BRDF Representation and Importance Sampling

Neural Global Illumination Via Superposed Deformable Feature Fields

Fast, Realistic Lighting and Material Design Using Nonlinear Cut Approximation

Error-aware Sampling in Adaptive Shells for Neural Surface Reconstruction

Adaptive Shells for Efficient Neural Radiance Field Rendering

BSDF Importance Baking: A Lightweight Neural Solution to Importance Sampling General Parametric BSDFs

RL-based Stateful Neural Adaptive Sampling and Denoising for Real-Time Path Tracing

Differentiable Rendering with Reparameterized Volume Sampling

Where and How: Mitigating Confusion in Neural Radiance Fields from Sparse Inputs

MadNIS -- Neural Multi-Channel Importance Sampling

Neural Radiance Flow for 4D View Synthesis and Video Processing

Neural Two-Level Monte Carlo Real-Time Rendering

Efficient Neural Radiance Fields with Learned Depth-Guided Sampling

Neural Relightable Participating Media Rendering

Fast Neural Scene Flow

Entropy-based Training Methods for Scalable Neural Implicit Sampler

AdaNeRF: Adaptive Sampling for Real-time Rendering of Neural Radiance Fields

Neural Control Variates

Importance Corrected Neural JKO Sampling