Abstract:Although neural supersampling has achieved great success in various applications for improving image quality, it is still difficult to apply it to a wide range of real-time rendering applications due to the high computational power demand. Most existing methods are computationally expensive and require high-performance hardware, preventing their use on platforms with limited hardware, such as smartphones. To this end, we propose a new supersampling framework for real-time rendering applications to reconstruct a high-quality image out of a low-resolution one, which is sufficiently lightweight to run on smartphones within a real-time budget. Our model takes as input the renderer-generated low resolution content and produces high resolution and anti-aliased results. To maximize sampling efficiency, we propose using an alternate sub-pixel sample pattern during the rasterization process. This allows us to create a relatively small reconstruction model while maintaining high image quality. By accumulating new samples into a high-resolution history buffer, an efficient history check and re-usage scheme is introduced to improve temporal stability. To our knowledge, this is the first research in pushing real-time neural supersampling on mobile devices. Due to the absence of training data, we present a new dataset containing 57 training and test sequences from three game scenes. Furthermore, based on the rendered motion vectors and a visual perception study, we introduce a new metric called inter-frame structural similarity (IF-SSIM) to quantitatively measure the temporal stability of rendered videos. Extensive evaluations demonstrate that our supersampling model outperforms existing or alternative solutions in both performance and temporal stability.

Temporally Stable Real-Time Joint Neural Denoising and Supersampling

Grouped Multi-Scale Network for Real-World Image Denoising.

Neural supersampling for real-time rendering

Joint Optimization of Triangle Mesh, Material, and Light from Neural Fields with Neural Radiance Cache

Neural Super-Resolution in Real-Time Rendering Using Auxiliary Feature Enhancement

RL-based Stateful Neural Adaptive Sampling and Denoising for Real-Time Path Tracing

Efficient neural supersampling on a novel gaming dataset

Neural Temporal Denoising for Indirect Illumination

Neural Super-Resolution for Real-time Rendering with Radiance Demodulation

MNSS: Neural Supersampling Framework for Real-Time Rendering on Mobile Devices.

Adversarial Monte Carlo denoising with conditioned auxiliary feature modulation

Real-time Monte Carlo Denoising with Weight Sharing Kernel Prediction Network

High-Quality Real-Time Rendering Using Subpixel Sampling Reconstruction

Fast Local Neural Regression for Low-Cost, Path Traced Lambertian Global Illumination

ExtraNet: Real-time Extrapolated Rendering for Low-latency Temporal Supersampling

Deep-Learning Realtime Upsampling Techniques in Video Games

Neural Denoising for Path Tracing of Medical Volumetric Data

Non-homogeneous denoising for virtual reality in real-time path tracing rendering

Low-latency Space-time Supersampling for Real-time Rendering

Progressive Denoising of Monte Carlo Rendered Images

Real-Time Neural Appearance Models