Abstract:Modern graphics processing units (GPUs) are delivering tremendous computing horsepower by running tens of thousands of threads concurrently. The massively parallel execution model has been effective to hide the long latency of off-chip memory accesses in graphics and other general computing applications exhibiting regular memory behaviors. With the fast-growing demand for general purpose computing on GPUs (GPGPU), GPU workloads are becoming highly diversified, and thus requiring a synergistic coordination of both computing and memory resources to unleash the computing power of GPUs. Accordingly, recent graphics processors begin to integrate an on-die level-2 (L2) cache. The huge number of threads on GPUs, however, poses significant challenges to L2 cache design. The experiments on a variety of GPGPU applications reveal that the L2 cache may or may not improve the overall performance depending on the characteristics of applications. In this paper, we propose efficient techniques to improve GPGPU performance by orchestrating both L2 cache and memory in a unified framework. The basic philosophy is to exploit the temporal locality among the massive number of concurrent memory requests and minimize the impact of memory divergence behaviors among simultaneously executed groups of threads. Our major contributions are twofold. First, a priority-based cache management is proposed to maximize the chance of frequently revisited data to be kept in the cache. Second, an effective memory scheduling is introduced to reorder memory requests in the memory controller according to the divergence behavior for reducing average waiting time of warps. Simulation results reveal that our techniques enhance the overall performance by 10% on average for memory intensive benchmarks, whereas the maximum gain can be up to 30%.

Group Scheduling for Improving Both CPU and Memory Power Efficiency Simultaneously

Power-Aware Buddy System and Task Group Scheduler

Coordinate page allocation and thread group for improving main memory power efficiency

A Way to Improve the Performance of Integrated Real-Time Monitoring System in Power Grid by Distributed Shared Memory Technology

Share Memory Aware Scheduler

Memory Affinity: Balancing Performance, Power, Thermal and Fairness for Multi-core Systems

Design Of A Dynamic Memory Access Scheduler

High-Performance and Energy-Effcient Memory Scheduler Design for Heterogeneous Systems

Process scheduling-based energy management for bank-aware memory system

Process Scheduling Based Memory Energy Management For Multi-Core Mobile Devices

Energy Aware Loop Scheduling for High Performance Multi-Module Memory

Priority Based Fair Scheduling : A Memory Scheduler Design for Chip-Multiprocessor Systems

Energy Efficient Real-Time Task Scheduling for Embedded Systems with Hybrid Main Memory

Memory Power Optimization on Different Memory Address Mapping Schemas.

Thread Batching for High-performance Energy-efficient GPU Memory Design

Orchestrating Cache Management and Memory Scheduling for GPGPU Applications.

A perceptual and predictive batch-processing memory scheduling strategy for a CPU-GPU heterogeneous system

Memory Access Scheduling Based on Dynamic Multilevel Priority in Shared DRAM Systems

Memory Power Optimisation on Low-Bit Multi-Access Cross Memory Address Mapping Schema

Frequency Affinity: Analyzing and Maximizing Power Efficiency in Multi-core Systems

Joint Modulo Scheduling And Memory Partitioning With Multi-Bank Memory For High-Level Synthesis