Abstract:The increasing demand for computing power and the emergence of heterogeneous computing architectures have driven the exploration of innovative techniques to address current limitations in both the compute and memory subsystems. One such solution is the use of Accelerated Processing Units (APUs), processors that incorporate both a central processing unit (CPU) and an integrated graphics processing unit (iGPU). However, the performance of both APU and CPU systems can be significantly hampered by address translation overhead, leading to a decline in overall performance, especially for cache-resident workloads. To address this issue, we propose the introduction of a new intermediate address space (IAS) in both APU and CPU systems. IAS serves as a bridge between virtual address (VA) spaces and physical address (PA) spaces, optimizing the address translation process. In the case of APU systems, our research indicates that the iGPU suffers from significant translation look-aside buffer (TLB) misses in certain workload situations. Using an IAS, we can divide the initial address translation into front- and back-end phases, effectively shifting the bottleneck in address translation from the cache side to the memory controller side, a technique that proves to be effective for cache-resident workloads. Our simulations demonstrate that implementing IAS in the CPU system can boost performance by up to 40% compared to conventional CPU systems. Furthermore, we evaluate the effectiveness of APU systems, comparing the performance of IAS-based systems with traditional systems, showing up to a 185% improvement in APU system performance with our proposed IAS implementation. Furthermore, our analysis indicates that over 90% of TLB misses can be filtered by the cache, and employing a larger cache within the system could potentially result in even greater improvements. The proposed IAS offers a promising and practical solution to enhance the performance of both APU and CPU systems, contributing to state-of-the-art research in the field of computer architecture.

In-memory Key-value Cache System Based on APUs

Put an Elephant into a Fridge

A design framework for processing-in-memory accelerator

Orchestrating Cache Management and Memory Scheduling for GPGPU Applications.

A Sharing-Aware L1.5D Cache for Data Reuse in GPGPUs

Performance evaluation and optimization of cache on fused CPU-GPU architecture

Coordinated Static and Dynamic Cache Bypassing for GPUs

DIDO: Dynamic Pipelines for In-Memory Key-Value Stores on Coupled CPU-GPU Architectures

Advanced hybrid MRAM based novel GPU cache system for graphic processing with high efficiency

Intermediate Address Space: virtual memory optimization of heterogeneous architectures for cache-resident workloads

A Distributed In-Memory Key-Value Store System on Heterogeneous CPU–GPU Cluster

POSTER: Accelerate GPU Concurrent Kernel Execution by Mitigating Memory Pipeline Stalls

Atomic Cache: Enabling Efficient Fine-Grained Synchronization with Relaxed Memory Consistency on GPGPUs Through In-Cache Atomic Operations

Understanding the Tradeoffs Between Software-Managed Vs. Hardware-Managed Caches in GPUs

Re-Cache: Mitigating Cache Contention by Exploiting Locality Characteristics with Reconfigurable Memory Hierarchy for GPGPUs.

Locality-Driven Dynamic Gpu Cache Bypassing

Optimizing Cache Bypassing and Warp Scheduling for GPUs

Exploring Cache Bypassing and Partitioning for Multi-Tasking on GPUs

An Enhanced Data Cache with In-Cache Processing Units for Convolutional Neural Network Accelerators

IBOM: an Integrated and Balanced On-Chip Memory for High Performance GPGPUs

An Efficient Compiler Framework for Cache Bypassing on GPUs