Abstract:Modern graphics processing units (GPUs) are delivering tremendous computing horsepower by running tens of thousands of threads concurrently. The massively parallel execution model has been effective to hide the long latency of off-chip memory accesses in graphics and other general computing applications exhibiting regular memory behaviors. With the fast-growing demand for general purpose computing on GPUs (GPGPU), GPU workloads are becoming highly diversified, and thus requiring a synergistic coordination of both computing and memory resources to unleash the computing power of GPUs. Accordingly, recent graphics processors begin to integrate an on-die level-2 (L2) cache. The huge number of threads on GPUs, however, poses significant challenges to L2 cache design. The experiments on a variety of GPGPU applications reveal that the L2 cache may or may not improve the overall performance depending on the characteristics of applications. In this paper, we propose efficient techniques to improve GPGPU performance by orchestrating both L2 cache and memory in a unified framework. The basic philosophy is to exploit the temporal locality among the massive number of concurrent memory requests and minimize the impact of memory divergence behaviors among simultaneously executed groups of threads. Our major contributions are twofold. First, a priority-based cache management is proposed to maximize the chance of frequently revisited data to be kept in the cache. Second, an effective memory scheduling is introduced to reorder memory requests in the memory controller according to the divergence behavior for reducing average waiting time of warps. Simulation results reveal that our techniques enhance the overall performance by 10% on average for memory intensive benchmarks, whereas the maximum gain can be up to 30%.

Global Memory Management in Distributed Multimedia Storage Systems

L-QoCo: learning to optimize cache capacity overloading in storage systems

Network-Adaptive Cache Management Schemes for Mixed Media

Implementing Cooperative Caching in Distributed Streaming Media Server Clusters

Efficient Distributed Memory Management with RDMA and Caching

Management of the Last Level Cache for Multimedia Application System

Scheme for Improving the Performance of Multimedia Storage Systems

A data affinity based garbage collector for multi-bank flash-memory storage system

Synchronization-Oriented Placement and Retrieval Strategies for Delay-Sensitive Media Streams

Adaptive Energy-Aware Design Of A Multi-Bank Flash-Memory Storage System

A Cache Replacement Algorithm in Hierarchical Storage of Continuous Media Object

Efficient storage management for aged file systems on persistent memory.

GCS: Collaborative video cache management strategy in multi-access edge computing

Megalloc: Fast Distributed Memory Allocator for NVM-Based Cluster

Using Memcached to Promote Read Throughput in Massive Small-File Storage System

Caching with Time Domain Buffer Sharing

A novel non-volatile memory storage system for I/O-intensive applications

Efficient and Consistent NVMM Cache for SSD-Based File System

Orchestrating Cache Management and Memory Scheduling for GPGPU Applications.

Improving In-Memory File System Reading Performance by Fine-Grained User-Space Cache Mechanisms

MAC: a novel systematically multilevel cache replacement policy for PCM memory