What problem does this paper attempt to address?

### Problems Addressed by the Paper The paper aims to address the high computational cost and time consumption issues faced when generating high-quality, high-resolution rendered images in real-time. Specifically, traditional physically-based rendering methods typically require multiple samples per pixel (samples-per-pixel, spp), leading to high computational costs and long rendering times. While reducing the number of samples per pixel can speed up rendering, it results in noticeable noise in the images. Existing denoising methods have alleviated this issue to some extent but have not yet achieved real-time performance at high resolutions. To overcome these challenges, the paper proposes a new Monte Carlo sampling strategy—subpixel sampling—and a corresponding denoising network—subpixel sampling reconstruction (SSR). Using these methods, the paper achieves the generation of high-quality rendered images at 2K resolution at a speed of 130 frames per second (FPS). ### Main Contributions 1. **Subpixel Sampling Strategy**: By dividing each frame into non-overlapping 2×2 blocks and computing only one ray-traced pixel per block, the sampling time for physically-based rendering is significantly reduced to one-third of the original. 2. **Subpixel Sampling Reconstruction Network (SSR)**: This network consists of two parts: a temporal feature accumulator and a reconstruction network. The temporal feature accumulator improves the perceptual range of pixels through temporal accumulation, while the reconstruction network uses a multi-scale U-Net structure to reconstruct high-quality image sequences. 3. **Superior Performance**: Experimental results show that this method significantly outperforms existing methods in terms of denoising quality and overall time cost, achieving real-time reconstruction performance at 2K resolution for the first time. ### Method Overview 1. **Subpixel Sampling**: - **1/4-spp Mode**: Each frame is divided into 2×2 blocks, and only one pixel's ray-tracing result is computed per block. - **GBuffer**: The rasterization pipeline is used to efficiently output high-resolution GBuffer, including features such as color, normals, and transparency. - **Mask Map**: A mask map is generated to mark sampled and unsampled pixels, guiding the weight prediction of the temporal feature accumulator. - **Demodulation**: The RGB image is demodulated using albedo and remodulated after reconstruction. 2. **Subpixel Sampling Reconstruction (SSR)**: - **Temporal Feature Accumulator**: Two convolutional networks process the current frame and the previous frame's features separately, calculating the blending factors for temporal stable feature accumulation. - **Reconstruction Network**: An extended U-Net structure is used to predict two coarse-scale images instead of dense features during the decoding stage, speeding up inference and improving image quality. ### Experimental Results - **Quantitative Evaluation**: In a test set of six different scenes, this method performs excellently in terms of PSNR and SSIM metrics, significantly outperforming existing methods. - **Qualitative Evaluation**: Visual comparisons show that this method generates high-quality images in all scenes, particularly excelling in handling sparse sampling data at 1/4-spp. - **Time Analysis**: The subpixel sampling strategy significantly reduces sampling time from 12.79 milliseconds to 4.35 milliseconds, and the total rendering time from 15.36 milliseconds to 6.92 milliseconds. The SSR network achieves 130 FPS at 2K resolution and 220 FPS at 1080p resolution. ### Conclusion The paper proposes a new subpixel sampling strategy and a corresponding denoising network, successfully addressing the computational cost and time consumption issues in real-time generation of high-quality, high-resolution rendered images. Experimental results show that this method outperforms existing methods in both denoising quality and real-time performance, with significant application prospects. Future work will focus on further optimizing inference time and accuracy, as well as exploring more application scenarios.

High-Quality Real-Time Rendering Using Subpixel Sampling Reconstruction

Neural Super-Resolution in Real-Time Rendering Using Auxiliary Feature Enhancement

Adaptive Sampling and Reconstruction for Gradient-Domain Rendering

RL-based Stateful Neural Adaptive Sampling and Denoising for Real-Time Path Tracing

Real-time Monte Carlo Denoising with Weight Sharing Kernel Prediction Network

Real-time Realistic Volume Rendering of Consistently High Quality with Dynamic Illumination

Adversarial Monte Carlo denoising with conditioned auxiliary feature modulation

Neural supersampling for real-time rendering

Adaptive Incident Radiance Field Sampling and Reconstruction Using Deep Reinforcement Learning.

Efficient Deep Models for Real-Time 4K Image Super-Resolution. NTIRE 2023 Benchmark and Report.

Low-latency Space-time Supersampling for Real-time Rendering

Point-Based Neural Scene Rendering for Street Views

Efficient Depth-Of-Field Rendering with Adaptive Sampling and Multiscale Reconstruction

Neural Super-Resolution for Real-time Rendering with Radiance Demodulation

AdaNeRF: Adaptive Sampling for Real-time Rendering of Neural Radiance Fields

Real‐time Denoising Using BRDF Pre‐integration Factorization

Real-Time Monte Carlo Denoising With Adaptive Fusion Network

MNSS: Neural Supersampling Framework for Real-Time Rendering on Mobile Devices.

Efficient Ray Sampling for Radiance Fields Reconstruction

Two-Stage Monte Carlo Denoising with Adaptive Sampling and Kernel Pool

Progressive Denoising of Monte Carlo Rendered Images