Abstract:Although neural supersampling has achieved great success in various applications for improving image quality, it is still difficult to apply it to a wide range of real-time rendering applications due to the high computational power demand. Most existing methods are computationally expensive and require high-performance hardware, preventing their use on platforms with limited hardware, such as smartphones. To this end, we propose a new supersampling framework for real-time rendering applications to reconstruct a high-quality image out of a low-resolution one, which is sufficiently lightweight to run on smartphones within a real-time budget. Our model takes as input the renderer-generated low resolution content and produces high resolution and anti-aliased results. To maximize sampling efficiency, we propose using an alternate sub-pixel sample pattern during the rasterization process. This allows us to create a relatively small reconstruction model while maintaining high image quality. By accumulating new samples into a high-resolution history buffer, an efficient history check and re-usage scheme is introduced to improve temporal stability. To our knowledge, this is the first research in pushing real-time neural supersampling on mobile devices. Due to the absence of training data, we present a new dataset containing 57 training and test sequences from three game scenes. Furthermore, based on the rendered motion vectors and a visual perception study, we introduce a new metric called inter-frame structural similarity (IF-SSIM) to quantitatively measure the temporal stability of rendered videos. Extensive evaluations demonstrate that our supersampling model outperforms existing or alternative solutions in both performance and temporal stability.

Superpixel-based Efficient Sampling for Learning Neural Fields from Large Input

Efficient Ray Sampling for Radiance Fields Reconstruction

Neural supersampling for real-time rendering

Point-Based Neural Scene Rendering for Street Views

Neural Super-Resolution in Real-Time Rendering Using Auxiliary Feature Enhancement

Efficient neural supersampling on a novel gaming dataset

SCSNet: an Efficient Paradigm for Learning Simultaneously Image Colorization and Super-resolution

MNSS: Neural Supersampling Framework for Real-Time Rendering on Mobile Devices.

Accelerating Neural Field Training via Soft Mining

Neural Global Illumination Via Superposed Deformable Feature Fields

Efficient Neural Radiance Fields with Learned Depth-Guided Sampling

High-Quality Real-Time Rendering Using Subpixel Sampling Reconstruction

SG-NeRF: Sparse-Input Generalized Neural Radiance Fields for Novel View Synthesis

ZS-SRT: An Efficient Zero-Shot Super-Resolution Training Method for Neural Radiance Fields

Learning-based Ray Sampling Strategy for Computation Efficient Neural Radiance Field Generation

Accelerated Neural Radiation Field Based on Optimal Ray Sampling

Fast Learning Radiance Fields by Shooting Much Fewer Rays

Super-Resolution Neural Radiance Field via Learning High Frequency Details for High-Fidelity Novel View Synthesis

ExtraNet: Real-time Extrapolated Rendering for Low-latency Temporal Supersampling

SamplingAug: on the Importance of Patch Sampling Augmentation for Single Image Super-Resolution.

Improving Neural Radiance Field using Near-Surface Sampling with Point Cloud Generation