Abstract:As the applications of Augmented Reality (AR) or Virtual Reality (VR) expand rapidly with the growing demands on enhanced visual realism, photorealistic image generation and insertion has become an essential feature for the emerging AR applications providing real-time workplace/household visual assistance. Physical Based Ray-Tracing (PBRT) is often used where synthesized images are generated by simulating the real environment and tracing the light transportation to achieve photorealistic effects, such as reflection, refraction, soft shadows, etc. PBRT is widely used in product design, medical visualization, video games and movie effects. To enable photorealistic rendering, there is a strong demand to support ray-tracing (RT) on mobile devices [1]. However, the challenges are: (1) unstructured memory access pattern and complex control flow lead to scheduling difficulty; (2) high memory requirements exhaust the limited SRAM space on edge devices; (3) low error tolerance requires high precision for computing; (4) complex computations, such as division and square root, require significant computing resources for the edge devices. As a result, common rendering engines such as Apple ARKit, OpenGL, are mainly based on the lower cost rasterization rendering technique. Unfortunately, rasterization rendering fails to produce photorealistic synthesis as shown in Fig. 2.5.1. Few ASICs have been fabricated so far as a mobile photorealistic rendering solution solution, however, they may not support RT [2], or may suffer from low efficiency [3]. This work has developed a ray-tracing processor, which also supports inverse rendering (IR) for background extraction [4]. The key features of this work include: (1) an ASIC rendering processor that embeds an end-to-end PBRT solution with IR for AR on mobile devices, (2) a reconfigurable mixed-precision PE design supporting diverse computing tasks for both IR and RT, (3) background clustered Field of View (FOV)-focused 3D construction reducing conventional background scene complexity from O(nlogn) to O(1), (4) scalable partitioning scheme for complex 3D objects, with an average of $13 \times$ speed up on test scenes, (5) use of Global RT Scheduler (GRTS) and Global Memory Access Controller (GMAC) to overcome the challenges of irregular memory access pattern and varied PE run-time with overall $684 \times$ speedup compared with the baseline design. The 28nm test chip achieves $3.95 - 28.8 \times$ higher rendering efficiency compared with existing ASIC solutions, enabling real-time PBRT rendering on mobile edge devices.

RTA: an Efficient SIMD Architecture for Ray Tracing

RT Engine: an Efficient Hardware Architecture for Ray Tracing

Efficient Data Management for Incoherent Ray Tracing.

A Fast Sah-Based Construction of Octree

Incoherent Ray tracing on GPU.

Realtime Ray Tracing on a Hibrid Parallel Architecture

Toward Real-Time Ray Tracing

Toward Real-Time Ray Tracing: A Survey on Hardware Acceleration and Microarchitecture Techniques.

MPRTA: an Efficient Multilevel Parallel Mobile Accelerator for High-Performance Ray Tracing

MMsRT: A Hardware Architecture for Ray Tracing in the Mobile Domain.

Efficient Kd-Tree Construction for Ray Tracing Using Ray Distribution Sampling

A Fully Pipelined KD-Tree Traversal Hardware Architecture for Ray Tracing

FastTree: a Hardware KD-tree Construction Acceleration Engine for Real-Time Ray Tracing

Agglomerative Memory and Thread Scheduling for High-Performance Ray-Tracing on GPUs

OpenCL-Based Real-Time KD-Tree and Raytracing for Dynamic Scene

Using Hardware Ray Transforms to Accelerate Ray/Primitive Intersections for Long, Thin Primitive Types.

A Framework for Quantum Ray Tracing

Tile-Based Ray Tracing on GPU

2.5 A 28nm Physical-Based Ray-Tracing Rendering Processor for Photorealistic Augmented Reality with Inverse Rendering and Background Clustering for Mobile Devices

RTScan: Efficient Scan with Ray Tracing Cores

Implementation and Optimization of Hardware-Universal Ray-tracing Underlying Algorithm Based on GPU Programming