Abstract:Differentiable rendering is a technique used in an important emerging class of visual computing applications that involves representing a 3D scene as a model that is trained from 2D images using gradient descent. Recent works (e.g. 3D Gaussian Splatting) use a rasterization pipeline to enable rendering high quality photo-realistic imagery at high speeds from these learned 3D models. These methods have been demonstrated to be very promising, providing state-of-art quality for many important tasks. However, training a model to represent a scene is still a time-consuming task even when using powerful GPUs. In this work, we observe that the gradient computation phase during training is a significant bottleneck on GPUs due to the large number of atomic operations that need to be processed. These atomic operations overwhelm atomic units in the L2 partitions causing stalls. To address this challenge, we leverage the observations that during the gradient computation: (1) for most warps, all threads atomically update the same memory locations; and (2) warps generate varying amounts of atomic traffic (since some threads may be inactive). We propose DISTWAR, a software-approach to accelerate atomic operations based on two key ideas: First, we enable warp-level reduction of threads at the SM sub-cores using registers to leverage the locality in intra-warp atomic updates. Second, we distribute the atomic computation between the warp-level reduction at the SM and the L2 atomic units to increase the throughput of atomic computation. Warps with many threads performing atomic updates to the same memory locations are scheduled at the SM, and the rest using L2 atomic units. We implement DISTWAR using existing warp-level primitives. We evaluate DISTWAR on widely used raster-based differentiable rendering workloads. We demonstrate significant speedups of 2.44x on average (up to 5.7x).

DAW-DMR: Divergence-Aware Warped DMR with Full Error Detection for GPGPU S

Improving branch divergence performance on GPGPU with a new PDOM stack and multi-level warp scheduling.

An Accurate GPU Performance Model for Effective Control Flow Divergence Optimization.

Improve GPGPU Latency Hiding with a Hybrid Recovery Stack and a Window Based Warp Scheduling Policy.

Enabling Software Resilience in GPGPU Applications via Partial Thread Protection

An Accurate Gpu Performance Model For Effective Control Flow Divergence Optimization

LWSDP: Locality-Aware Warp Scheduling and Dynamic Data Prefetching Co-design in the Per-SM Private Cache of GPGPUs

Warp-Aware Adaptive Energy Efficiency Calibration for Multi-GPU Systems

DISTWAR: Fast Differentiable Rendering on Raster-based Rendering Pipelines

Towards Accelerating Irregular EDA Applications with GPUs.

A GPU parallel scheme for accelerating 2D and 3D peridynamics models

Taming irregular EDA applications on GPUs.

Accelerating Adaptive IDW Interpolation Algorithm on a Single GPU

PRODA: Improving Parallel Programs on GPUs Through Dependency Analysis

Dynamically Reconfigurable Memory Address Mapping for General-Purpose Graphics Processing Unit.

DGR: Tackling Drifted and Correlated Noise in Quantum Error Correction via Decoding Graph Re-weighting

Accelerating DEM simulations on GPUs by reducing the impact of warp divergences

An optimized large-scale hybrid DGEMM design for CPUs and ATI GPUs.

A Low-Power Hardware-Friendly Optimisation Algorithm With Absolute Numerical Stability and Convergence Guarantees

Adaptive Data Path Selection for Durable Transaction in GPU Persistent Memory

Accelerating Image-Domain-Warping Virtual View Synthesis on GPGPU