Abstract:It has been shown that rendering in the gradient domain, i.e., estimating finite difference gradients of image intensity using correlated samples, and combining them with direct estimates of pixel intensities by solving a screened Poisson problem, often offers fundamental benefits over merely sampling pixel intensities. The reasons can be traced to the frequency content of the light transport integrand and its interplay with the gradient operator. However, while they often yield state of the art performance among algorithms that are based on Monte Carlo sampling alone, gradient-domain rendering algorithms have, until now, not generally been competitive with techniques that combine Monte Carlo sampling with post-hoc noise removal using sophisticated non-linear filtering. Drawing on the power of modern convolutional neural networks, we propose a novel reconstruction method for gradient-domain rendering. Our technique replaces the screened Poisson solver of previous gradient-domain techniques with a novel dense variant of the U-Net autoencoder, additionally taking auxiliary feature buffers as inputs. We optimize our network to minimize a perceptual image distance metric calibrated to the human visual system. Our results significantly improve the quality obtained from gradient-domain path tracing, allowing it to overtake state-of-the-art comparison techniques that denoise traditional Monte Carlo samplings. In particular, we observe that the correlated gradient samples --- that offer information about the smoothness of the integrand unavailable in standard Monte Carlo sampling --- notably improve image quality compared to an equally powerful neural model that does not make use of gradient samples.

Filtering-Based Reconstruction for Gradient-Domain Rendering

Adaptive Sampling and Reconstruction for Gradient-Domain Rendering

Robust surface reconstruction from gradient fields

Deep convolutional reconstruction for gradient-domain rendering

L0 image smoothing via iterating truncated L1 gradient regularization

Unsupervised Image Reconstruction for Gradient‐Domain Volumetric Rendering

Robust Color Gradient Estimation for Photographic Volumes.

Gradient Estimation for Real-time Adaptive Temporal Filtering

A Novel Robust Color Gradient Estimator for Photographic Volume Visualization

Multiview Textured Mesh Recovery by Differentiable Rendering

A Gradient-Domain-Based Edge-Preserving Sharpen Filter

Gradient Domain Weighted Guided Image Filtering

Edge-Aware Volume Smoothing Using L0 Gradient Minimization

A Gradient-Based Adaptive Interpolation Filter for Multiple View Synthesis

Using Adaptive Gradient for Texture Learning in Single-View 3D Reconstruction

Efficient Differentiation of Pixel Reconstruction Filters for Path-Space Differentiable Rendering

A Nonlinear Gradient Domain-Guided Filter Optimized by Fractional-Order Gradient Descent with Momentum RBF Neural Network for Ship Image Dehazing

Faster Gradient Descent and the Efficient Recovery of Images

Lossless Basis Expansion for Gradient‐Domain Rendering

Gradient-domain image reconstruction framework using chroma-preserving intensity-range and base-structure constraints

Spatio-spectral Image Reconstruction Using Non-local Filtering