Abstract:Following the advances in technology scaling, graphics processing units (GPUs) incorporate an increasing amount of computing resources and it becomes difficult for a single GPU kernel to fully utilize the vast GPU resources. One solution to improve resource utilization is concurrent kernel execution (CKE). Early CKE mainly targets the leftover resources. However, it fails to optimize the resource utilization and does not provide fairness among concurrent kernels. Spatial multitasking assigns a subset of streaming multiprocessors (SMs) to each kernel. Although achieving better fairness, the resource underutilization within an SM is not addressed. Thus, intra-SM sharing has been proposed to issue thread blocks from different kernels to each SM. However, as shown in this study, the overall performance may be undermined in the intra-SM sharing schemes due to the severe interference among kernels. Specifically, as concurrent kernels share the memory subsystem, one kernel, even as computing-intensive, may starve from not being able to issue memory instructions in time. Besides, severe L1 D-cache thrashing and memory pipeline stalls caused by one kernel, especially a memory-intensive one, will impact other kernels, further hurting the overall performance. In this study, we investigate various approaches to overcome the aforementioned problems exposed in intra-SM sharing. We first highlight that cache partitioning techniques proposed for CPUs are not effective for GPUs. Then we propose two approaches to reduce memory pipeline stalls. The first is to balance memory accesses of concurrent kernels. The second is to limit the number of inflight memory instructions issued from individual kernels. Our evaluation shows that the proposed schemes significantly improve the weighted speedup of two state-of-the-art intra-SM sharing schemes, Warped-Slicer and SMK, by 24.6% and 27.2% on average, respectively, with lightweight hardware overhead.

Accelerating Large Scale Image Analyses on Parallel, CPU-GPU Equipped Systems

High-throughput Analysis of Large Microscopy Image Datasets on CPU-GPU Cluster Platforms

Efficient Execution of Microscopy Image Analysis on CPU, GPU, and MIC Equipped Cluster Systems

Parallel Processing of Massive Remote Sensing Images in a GPU Architecture.

Balancing Cpu And Gpu: Real-Time Visualization Of Large Scale 3d Scanning Models

Hybrid CPU-GPU Framework for Network Motifs

Accelerating Spatial Cross-Matching on CPU-GPU Hybrid Platform With CUDA and OpenACC

Adaptive Pipeline Parallelism for Image Feature Extraction Algorithms

Accelerating Spaceborne SAR Imaging Using Multiple CPU/GPU Deep Collaborative Computing

High Performance Computing Via a GPU

High Performance Graph Analytics with Productivity on Hybrid CPU-GPU Platforms

Rapid prototyping of image processing workflows on massively parallel architectures

Dual Buffer Rotation Four-Stage Pipeline for CPU–GPU Cooperative Computing

Exploiting the Task-Pipelined Parallelism of Stream Programs on Many-Core GPUs.

POSTER: Accelerate GPU Concurrent Kernel Execution by Mitigating Memory Pipeline Stalls

Accelerating Image Convolution Filtering Algorithms on Integrated CPU–GPU Architectures

Accelerating Large Scale Artificial Society Simulation with Cpu/Gpu Based Heterogeneous Parallel Method

Orchestrated Co-scheduling, Resource Partitioning, and Power Capping on CPU-GPU Heterogeneous Systems via Machine Learning

Design and Optimization of Remote Sensing Image Fusion Parallel Algorithms Based on CPU-GPU Heterogeneous Platforms

Heterogeneous Parallel Computing Accelerated Iterative Subpixel Digital Image Correlation

GPU-based acceleration of local binary pattern algorithm for Internet applications