Abstract:Modern graphics processing units (GPUs) are delivering tremendous computing horsepower by running tens of thousands of threads concurrently. The massively parallel execution model has been effective to hide the long latency of off-chip memory accesses in graphics and other general computing applications exhibiting regular memory behaviors. With the fast-growing demand for general purpose computing on GPUs (GPGPU), GPU workloads are becoming highly diversified, and thus requiring a synergistic coordination of both computing and memory resources to unleash the computing power of GPUs. Accordingly, recent graphics processors begin to integrate an on-die level-2 (L2) cache. The huge number of threads on GPUs, however, poses significant challenges to L2 cache design. The experiments on a variety of GPGPU applications reveal that the L2 cache may or may not improve the overall performance depending on the characteristics of applications. In this paper, we propose efficient techniques to improve GPGPU performance by orchestrating both L2 cache and memory in a unified framework. The basic philosophy is to exploit the temporal locality among the massive number of concurrent memory requests and minimize the impact of memory divergence behaviors among simultaneously executed groups of threads. Our major contributions are twofold. First, a priority-based cache management is proposed to maximize the chance of frequently revisited data to be kept in the cache. Second, an effective memory scheduling is introduced to reorder memory requests in the memory controller according to the divergence behavior for reducing average waiting time of warps. Simulation results reveal that our techniques enhance the overall performance by 10% on average for memory intensive benchmarks, whereas the maximum gain can be up to 30%.

Research on Cache Partitioning and Adaptive Replacement Policy for CPU-GPU Heterogeneous Processors

Performance Optimization by Dynamically Altering Cache Replacement Algorithm in CPU-GPU Heterogeneous Multi-Core Architecture.

Performance evaluation and optimization of cache on fused CPU-GPU architecture

Improving the Performance of Heterogeneous Multi-Core Processors by Modifying the Cache Coherence Protocol

A Heterogeneity-Aware Replacement Policy for the Partitioned Cache on Asymmetric Multi-Core Architectures

Understanding Data Partition for Applications on CPU-GPU Integrated Processors.

Miss-aware LLC Buffer Management Strategy Based on Heterogeneous Multi-Core

Adaptive Cache Management for Energy-Efficient GPU Computing.

Set variation-aware shared LLC management for CPU-GPU heterogeneous architecture

Exploring Time-Predictable and High-Performance Last-Level Caches for Hard Real-Time Integrated CPU-GPU Processors.

Partition-Based Cache Replacement to Manage Shared L2 Caches

Shared Last-Level Cache Management for GPGPUs with Hybrid Main Memory

LLC Buffer Management Strategy Based on Heterogeneous Multi-core

A perceptual and predictive batch-processing memory scheduling strategy for a CPU-GPU heterogeneous system

Orchestrating Cache Management and Memory Scheduling for GPGPU Applications.

Efficient Data Transfer in a Heterogeneous Multicore-Based CE Systems Using Cache Performance Optimization

Exploring Cache Bypassing and Partitioning for Multi-Tasking on GPUs

Dynamic Block Size Adjustment and Workload Balancing Strategy Based on CPU-GPU Heterogeneous Platform

Performance Optimization for CPU-GPU Heterogeneous Parallel System

SRAM- and STT-RAM-based Hybrid, Shared Last-Level Cache for On-Chip CPU–GPU Heterogeneous Architectures

A Memory Scheduling Strategy for Eliminating Memory Access Interference in Heterogeneous System