Abstract:Heterogeneous computing nodes are now pervasive throughout computing, and GPUs have emerged as a leading computing device for application acceleration. GPUs have tremendous computing potential for data-parallel applications, and the emergence of GPUs has led to proliferation of GPU-accelerated applications. This proliferation has also led to systems in which many applications are competing for access to GPU resources, and efficient utilization of the GPU resources is critical to system performance. Prior techniques of temporal multitasking can be employed with GPU resources as well, but not all GPU kernels make full use of the GPU resources. There is, therefore, an unmet need for spatial multitasking in GPUs. Resources used inefficiently by one kernel can be instead assigned to another kernel that can more effectively use the resources. In this paper we propose a software-hardware solution for efficient spatial-temporal multitasking and a software based emulation framework for our system. We pair an efficient heuristic in software with hardware leaky-bucket based thread-block interleaving to implement spatial-temporal multitasking. We demonstrate our techniques on various GPU architecture using nine representative benchmarks from CUDA SDK. Our experiments on Fermi GTX480 demonstrate performance improvement by up to 46% (average 26%) over sequential GPU task execution and 37% (average 18%) over default concurrent multitasking. Compared with the state-of-the-art Kepler K20 using Hyper-Q technology, our technique achieves up to 40% (average 17%) performance improvement over default concurrent multitasking.

SPGPU: Spatially Programmed GPU

Towards a GPU accelerated spatial computing framework

Efficient GPU Spatial-Temporal Multitasking

Exploiting the Task-Pipelined Parallelism of Stream Programs on Many-Core GPUs.

Parallel Transient Stability-Constrained Optimal Power Flow Using GPU as Coprocessor.

Improving GPU Performance Through Resource Sharing

Accelerating GPU Message Communication for Autonomous Navigation Systems.

Process variation-aware workload partitioning algorithms for GPUs supporting spatial-multitasking

GPU Memory Optimization Through Program Restructuring Methods

Enabling Efficient Spatio-Temporal GPU Sharing for Network Function Virtualization

LWSDP: Locality-Aware Warp Scheduling and Dynamic Data Prefetching Co-design in the Per-SM Private Cache of GPGPUs

NPGPU: Network processing on graphics processing units

Locality-aware Thread Block Design in Single and Multi-GPU Graph Processing

Efficient Resource Sharing Through GPU Virtualization on Accelerated High Performance Computing Systems

A Compiler-assisted Locality Aware CTA Mapping Scheme

GPU-based parallel particle swarm optimization

Exploiting Memory Access Patterns to Improve Memory Performance in Data-Parallel Architectures

The Landscape of GPU-Centric Communication

GPU Parallel Computing: Programming Language, Debugging Tools and Data Structures

A CPU-GPGPU Scheduler Based on Data Transmission Bandwidth of Workload

Performance analysis of a novel GPU computation-to-core mapping scheme for robust facet image modeling