Abstract:It has been shown that rendering in the gradient domain, i.e., estimating finite difference gradients of image intensity using correlated samples, and combining them with direct estimates of pixel intensities by solving a screened Poisson problem, often offers fundamental benefits over merely sampling pixel intensities. The reasons can be traced to the frequency content of the light transport integrand and its interplay with the gradient operator. However, while they often yield state of the art performance among algorithms that are based on Monte Carlo sampling alone, gradient-domain rendering algorithms have, until now, not generally been competitive with techniques that combine Monte Carlo sampling with post-hoc noise removal using sophisticated non-linear filtering. Drawing on the power of modern convolutional neural networks, we propose a novel reconstruction method for gradient-domain rendering. Our technique replaces the screened Poisson solver of previous gradient-domain techniques with a novel dense variant of the U-Net autoencoder, additionally taking auxiliary feature buffers as inputs. We optimize our network to minimize a perceptual image distance metric calibrated to the human visual system. Our results significantly improve the quality obtained from gradient-domain path tracing, allowing it to overtake state-of-the-art comparison techniques that denoise traditional Monte Carlo samplings. In particular, we observe that the correlated gradient samples --- that offer information about the smoothness of the integrand unavailable in standard Monte Carlo sampling --- notably improve image quality compared to an equally powerful neural model that does not make use of gradient samples.

Deep residual learning for denoising Monte Carlo renderings

Adversarial Monte Carlo denoising with conditioned auxiliary feature modulation

Denoising Deep Monte Carlo Renderings

A survey on deep learning-based Monte Carlo denoising

Enhancing and Learning Denoiser Without Clean Reference

DMCR-GAN: Adversarial Denoising for Monte Carlo Renderings with Residual Attention Networks and Hierarchical Features Modulation of Auxiliary Buffers

Progressive Denoising of Monte Carlo Rendered Images

Real-time Monte Carlo Denoising with Weight Sharing Kernel Prediction Network

Hierarchical Residual Learning for Image Denoising

Dilated Residual Encode-Decode Networks for Image Denoising

An Adaptive Image Denoising Method Based on Deep Rectified Denoising Auto-Encoder

Path‐based Monte Carlo Denoising Using a Three‐Scale Neural Network

Multi-Scale and Kernel-Predicting Convolutional Networks for Monte Carlo Denoising.

Multiplicative Denoising Method Based on Deep Residual Learning

ResDNN: deep residual learning for natural image denoising

A detail preserving neural network model for Monte Carlo denoising

DEMC: A Deep Dual-Encoder Network for Denoising Monte Carlo Rendering

Deep convolutional reconstruction for gradient-domain rendering

Denoising Monte Carlo renderings via a multi-scale featured dual-residual GAN

An Unsupervised Deep Learning Approach for Real-World Image Denoising.

Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising