Abstract:The unrivaled computing capabilities of modern GPUs meet the demand of processing massive amounts of data seen in many application domains. While traditional HPC systems support applications as standalone entities that occupy entire GPUs, there are GPU-based DBMSs where multiple tasks are meant to be run at the same time in the same device. To that end, system-level resource management mechanisms are needed to fully unleash the computing power of GPUs in large data processing, and there were some researches focusing on it. In our previous work, we explored the single compute-bound kernel modeling on GPUs under NVidia’s CUDA framework and provided an in-depth anatomy of the NVidia’s concurrent kernel execution mechanism (CUDA stream). This paper focuses on resource allocation of multiple GPU applications towards optimization of system throughput in the context of systems. Comparing to earlier studies of enabling concurrent tasks support on GPU such as MultiQx-GPU, we use a different approach that is to control the launching parameters of multiple GPU kernels as provided by compile-time performance modeling as a kernel-level optimization and also a more general pre-processing model with batch-level control to enhance performance. Specifically, we construct a variation of multi-dimensional knapsack model to maximize concurrency in a multi-kernel environment. We present an in-depth analysis of our model and develop an algorithm based on dynamic programming technique to solve the model. We prove the algorithm can find optimal solutions (in terms of thread concurrency) to the problem and bears pseudopolynomial complexity on both time and space. Such results are verified by extensive experiments running on our microbenchmark that consists of real-world GPU queries. Furthermore, solutions identified by our method also significantly reduce the total running time of the workload, as compared to sequential and MultiQx-GPU executions.

GPU computing using concurrent kernels: A case study

Kernel concurrency opportunities based on GPU benchmarks characterization

Concurrent query processing in a GPU-based database system

High Performance Computing Via a GPU

Two-Stage Modeling and Control of Concurrent Tasks in a Multi-Kernel GPGPU Environment

Exploiting Parallelism in the Simulation of General Purpose Graphics Processing Unit Program

An Empirical-cum-Statistical Approach to Power-Performance Characterization of Concurrent GPU Kernels

Performance evaluation and analysis of sparse matrix and graph kernels on heterogeneous processors

POSTER: Accelerate GPU Concurrent Kernel Execution by Mitigating Memory Pipeline Stalls

Performance Acceleration of Kernel Polynomial Method Applying Graphics Processing Units

Efficient GPU Spatial-Temporal Multitasking

Efficient Kernel Management on GPUs.

Acceleration for CFD Applications on Large GPU Clusters: an NPB Case Study

Fair and Cache Blocking Aware Warp Scheduling for Concurrent Kernel Execution on GPU

Accelerating GPGPU Architecture Simulation.

Low-Energy Kernel Scheduling Approach for Energy Saving.

A Scheme of Full Kinetic Particle-in-cell Algorithms for GPU Acceleration Using CUDA Fortran Programming

On Power-Performance Characterization of Concurrent Throughput Kernels

Multidisciplinary simulation acceleration using multiple shared memory graphical processing units

Kernel Scheduling Approach for Reducing GPU Energy Consumption

Simultaneous Multikernel GPU: Multi-tasking throughput processors via fine-grained sharing