Abstract:General-purpose graphics processing unit (GPGPU), widely recognized as an exceptional computing platform for de-ploying emerging parallel applications, requires strict adherence to atomicity and memory consistency models for shared variable synchronization. This is crucial to ensure deterministic execution and leverage the performance advantages of the GPGPU single-instruction -multiple-threads architecture. However, the escalating demand for shared variable updates across thread blocks, notably in applications like deep neural networks and graph analysis, significantly exacerbates the serialization overhead of atomic operations due to the von Neumann bottleneck. Additionally, the overhead introduced by memory fences supporting the memory consistency model further complicates this fine-grained synchronization requirement. To address these challenges, this paper proposes Atomic Cache, facilitating an In-Cache computing hardware-software co-design for GPGPUs. At the software level, we propose relaxed memory consistency based on non-ordering commutativity to alleviate the execution of in-cache atomic operations, thereby mitigating the performance overhead of memory fences. At the hardware level, we present the In-Situ Store Atomic Cache Macro, which empowers the Atomic Cache to efficiently execute atomic logic and arithmetic operations within the cache array. This innovation alleviates the von Neumann bottleneck associated with serialized execution of atomic operations. The experimental evaluation results demonstrate that the Atomic Cache can save more than 60% of memory access energy while incurring only 9.42% chip area overhead. Furthermore, it not only delivers an average speedup ratio of 2.59 × and an IPC performance improvement of 1.48× for RISC-V GPGPUs, but also achieves an average speedup ratio of 1.31 × and an IPC performance improvement of 39.92% when compared to state-of-the-art designs employing local atomic buffers.

Exploring Time-Predictable and High-Performance Last-Level Caches for Hard Real-Time Integrated CPU-GPU Processors.

Predictable Sharing of Last-level Cache Partitions for Multi-core Safety-critical Systems

Performance evaluation and optimization of cache on fused CPU-GPU architecture

Analyzing Memory Access on CPU-GPGPU Shared LLC Architecture

Buffer on Last Level Cache for CPU and GPGPU Data Sharing

Enable Back Memory and Global Synchronization on LLC Buffer

Last Level Cache Layout Remapping for Heterogeneous Systems

Orchestrating Cache Management and Memory Scheduling for GPGPU Applications.

Improving CPU and GPU Performance Through Sample-Based Dynamic LLC Bypassing

LLC Buffer for Arbitrary Data Sharing in Heterogeneous Systems.

LA-LLC: Inter-Core Locality-Aware Last-Level Cache to Exploit Many-to-Many Traffic in GPGPUs

WAP: the Warp Feature Aware Prefetching Method for LLC on CPU-GPU Heterogeneous Architecture

An Energy-Efficient Last Level Cache Design Using Software and Hardware Collaborative Region-based Management

A Sharing-Aware L1.5D Cache for Data Reuse in GPGPUs

Re-Cache: Mitigating Cache Contention by Exploiting Locality Characteristics with Reconfigurable Memory Hierarchy for GPGPUs.

Addressing Variability in Reuse Prediction for Last-Level Caches

Global Priority Table for Last-Level Caches

L2C2: Last-Level Compressed-Cache NVM and a Procedure to Forecast Performance and Lifetime

Cache Management with Partitioning-Aware Eviction and Thread-Aware Insertion/Promotion Policy

Atomic Cache: Enabling Efficient Fine-Grained Synchronization with Relaxed Memory Consistency on GPGPUs Through In-Cache Atomic Operations

Manage Last Level Cache with Eliminating Less Reused Blocks and Re-reference Interval Prediction Policy