Abstract:The continued growth of the computational capability of throughput processors has made throughput processors the platform of choice for a wide variety of high performance computing applications. Graphics Processing Units (GPUs) are a prime example of throughput processors that can deliver high performance for applications ranging from typical graphics applications to general-purpose data parallel (GPGPU) applications. However, this success has been accompanied by new performance bottlenecks throughout the memory hierarchy of GPU-based systems. We identify and eliminate performance bottlenecks caused by major sources of interference throughout the memory hierarchy. We introduce changes to the memory hierarchy for systems with GPUs that allow the memory hierarchy to be aware of both CPU and GPU applications' characteristics. We introduce mechanisms to dynamically analyze different applications' characteristics and propose four major changes throughout the memory hierarchy. We propose changes to the cache management and memory scheduling mechanisms to mitigate intra-application interference in GPGPU applications. We propose changes to the memory controller design and its scheduling policy to mitigate inter-application interference in heterogeneous CPU-GPU systems. We redesign the MMU and the memory hierarchy in GPUs to be aware of ddress-translation data in order to mitigate the inter-address-space interference. We introduce a hardware-software cooperative technique that modifies the memory allocation policy to enable large page support in order to further reduce the inter-address-space interference at the shared Translation Lookaside Buffer (TLB). Our evaluations show that the GPU-aware cache and memory management techniques proposed in this dissertation are effective at mitigating the interference caused by GPUs on current and future GPU-based systems.

Efficient Resource Sharing Through GPU Virtualization on Accelerated High Performance Computing Systems

A Virtual Multi-Channel GPU Fair Scheduling Method for Virtual Machines.

Exploring the Diversity of Multiple Job Deployments over GPUs for Efficient Resource Sharing

Parallel Transient Stability-Constrained Optimal Power Flow Using GPU as Coprocessor.

Efficient GPU Spatial-Temporal Multitasking

Implementation of GPU virtualization using PCI pass-through mechanism

Multi-Tenant Virtual GPUs for Optimising Performance of a Financial Risk Application

Enabling Efficient Spatio-Temporal GPU Sharing for Network Function Virtualization

Improving GPU Performance Through Resource Sharing

Simultaneous Multikernel GPU: Multi-tasking throughput processors via fine-grained sharing

Optimizing Hardware Resource Partitioning and Job Allocations on Modern GPUs under Power Caps

vCUDA: GPU accelerated high performance computing in virtual machines

Survey of GPU Virtualization

Enhanced GPU Resource Utilization through Fairness-aware Task Scheduling

High Performance Computing Via a GPU

Techniques for Shared Resource Management in Systems with Throughput Processors

Concurrent analytical query processing with GPUs

Cross-Core Data Sharing for Energy-Efficient GPUs

Orchestrated Co-scheduling, Resource Partitioning, and Power Capping on CPU-GPU Heterogeneous Systems via Machine Learning

ParvaGPU: Efficient Spatial GPU Sharing for Large-Scale DNN Inference in Cloud Environments

A Programming Framework Based on Multi-GPU