Abstract:Modern graphics processing units (GPUs) are delivering tremendous computing horsepower by running tens of thousands of threads concurrently. The massively parallel execution model has been effective to hide the long latency of off-chip memory accesses in graphics and other general computing applications exhibiting regular memory behaviors. With the fast-growing demand for general purpose computing on GPUs (GPGPU), GPU workloads are becoming highly diversified, and thus requiring a synergistic coordination of both computing and memory resources to unleash the computing power of GPUs. Accordingly, recent graphics processors begin to integrate an on-die level-2 (L2) cache. The huge number of threads on GPUs, however, poses significant challenges to L2 cache design. The experiments on a variety of GPGPU applications reveal that the L2 cache may or may not improve the overall performance depending on the characteristics of applications. In this paper, we propose efficient techniques to improve GPGPU performance by orchestrating both L2 cache and memory in a unified framework. The basic philosophy is to exploit the temporal locality among the massive number of concurrent memory requests and minimize the impact of memory divergence behaviors among simultaneously executed groups of threads. Our major contributions are twofold. First, a priority-based cache management is proposed to maximize the chance of frequently revisited data to be kept in the cache. Second, an effective memory scheduling is introduced to reorder memory requests in the memory controller according to the divergence behavior for reducing average waiting time of warps. Simulation results reveal that our techniques enhance the overall performance by 10% on average for memory intensive benchmarks, whereas the maximum gain can be up to 30%.

Adaptive VC Partitioning for NoCs in GPGPUs.

Power-and thermal-aware mapping for NoC under latency constraint

A low-cost conflict-free NoC for GPGPUs.

Enabling Efficient Spatio-Temporal GPU Sharing for Network Function Virtualization

A Virtual Multi-Channel GPU Fair Scheduling Method for Virtual Machines.

Understanding Data Partition for Applications on CPU-GPU Integrated Processors.

G-NET: Effective GPU Sharing in NFV Systems.

Optimizing Vertical Link Placement and Congestion Aware Dynamic Elevator Assignment for Partially Connected 3D-Nocs

An Access Pattern Based Adaptive Mapping Function for GPGPU Scratchpad Memory

Orchestrating Cache Management and Memory Scheduling for GPGPU Applications.

SPGPU: Spatially Programmed GPU

Automatic CNN Model Partitioning for GPU/FPGA-based Embedded Heterogeneous Accelerators using Geometric Programming

Optimizing Hardware Resource Partitioning and Job Allocations on Modern GPUs under Power Caps

DNN Partition and Offloading Strategy with Improved Particle Swarm Genetic Algorithm in VEC

Application-Aware Resource Allocation Based on Benefit–Cost Ratio in Computing Power Network with Heterogeneous Computing Resources

Exploring Time-Predictable and High-Performance Last-Level Caches for Hard Real-Time Integrated CPU-GPU Processors.

A hardware/software partitioning method based on graph convolution network

Towards Power Efficient DNN Accelerator Design on Reconfigurable Platform

An efficient GPU-based parallel tabu search algorithm for hardware/software co-design

A Parallel Strategy to Accelerate Neighborhood Operation for Raster Data Coordinating CPU and GPU

LWSDP: Locality-Aware Warp Scheduling and Dynamic Data Prefetching Co-design in the Per-SM Private Cache of GPGPUs