Abstract:Although neural supersampling has achieved great success in various applications for improving image quality, it is still difficult to apply it to a wide range of real-time rendering applications due to the high computational power demand. Most existing methods are computationally expensive and require high-performance hardware, preventing their use on platforms with limited hardware, such as smartphones. To this end, we propose a new supersampling framework for real-time rendering applications to reconstruct a high-quality image out of a low-resolution one, which is sufficiently lightweight to run on smartphones within a real-time budget. Our model takes as input the renderer-generated low resolution content and produces high resolution and anti-aliased results. To maximize sampling efficiency, we propose using an alternate sub-pixel sample pattern during the rasterization process. This allows us to create a relatively small reconstruction model while maintaining high image quality. By accumulating new samples into a high-resolution history buffer, an efficient history check and re-usage scheme is introduced to improve temporal stability. To our knowledge, this is the first research in pushing real-time neural supersampling on mobile devices. Due to the absence of training data, we present a new dataset containing 57 training and test sequences from three game scenes. Furthermore, based on the rendered motion vectors and a visual perception study, we introduce a new metric called inter-frame structural similarity (IF-SSIM) to quantitatively measure the temporal stability of rendered videos. Extensive evaluations demonstrate that our supersampling model outperforms existing or alternative solutions in both performance and temporal stability.

PowerNet: Learning-Based Real-Time Power-Budget Rendering

Real-time Rendering on a Power Budget.

On-the-Fly Power-Aware Rendering

Explore Training of Deep Convolutional Neural Networks on Battery-powered Mobile Devices: Design and Application

MOC: Multi-Objective Mobile CPU-GPU Co-Optimization for Power-Efficient DNN Inference

DACO: Pursuing Ultra-low Power Consumption Via DNN-Adaptive CPU-GPU CO-optimization on Mobile Devices

All-in-One: A Highly Representative DNN Pruning Framework for Edge Devices with Dynamic Power Management

Adaptive Recurrent Frame Prediction with Learnable Motion Vectors.

MetaSapiens: Real-Time Neural Rendering with Efficiency-Aware Pruning and Accelerated Foveated Rendering

PBR-Net: Imitating Physically Based Rendering using Deep Neural Network

NeuralPower: Predict and Deploy Energy-Efficient Convolutional Neural Networks

Mobile GPU Power Consumption Reduction Via Dynamic Resolution and Frame Rate Scaling.

RAVEN: Perception-aware Optimization of Power Consumption for Mobile Games.

Potamoi: Accelerating Neural Rendering via a Unified Streaming Architecture

Towards Real-Time Neural Volumetric Rendering on Mobile Devices: A Measurement Study

Display Power Reduction for Mobile Closed-Source Games.

Optimizing Power Consumption of Mobile Games.

BPNS: a Fast and Accurate Power Estimation Model for Embedded Devices

Content-Adaptive Display Power Saving for Internet Video Applications on Mobile Devices

Spiking-NeRF: Spiking Neural Network for Energy-Efficient Neural Rendering

MNSS: Neural Supersampling Framework for Real-Time Rendering on Mobile Devices.