Abstract:The abstraction of a shared memory space over separate CPU and GPU memory domains has eased the burden of portability for many HPC codebases. However, users pay for ease of use provided by system-managed memory with a moderate-to-high performance overhead. NVIDIA Unified Virtual Memory (UVM) is currently the primary real-world implementation of such abstraction and offers a functionally equivalent testbed for in-depth performance study for both UVM and future Linux Heterogeneous Memory Management (HMM) compatible systems. The continued advocacy for UVM and HMM motivates improvement of the underlying system. We focus on UVM-based systems and investigate root causes of UVM overhead, a non-trivial task due to complex interactions of multiple hardware and software constituents and the desired cost granularity. In our prior work, we delved deeply into UVM system architecture and showed internal behaviors of page fault servicing in batches. We provided quantitative evaluation of batch handling for various applications under different scenarios, including prefetching and oversubscription. We revealed the driver workload depends on the interactions among application access patterns, GPU hardware constraints, and host OS components. Host OS components have significant overhead present across implementations, warranting close attention. This extension furthers our prior study in three aspects: fine-grain cost analysis and breakdown, extension to multiple GPUs, and investigation of platforms with different GPU-GPU interconnects. We take a top-down approach to quantitative batch analysis and uncover how constituent component costs accumulate and overlap governed by synchronous and asynchronous operations. Our multi-GPU analysis shows reduced cost of GPU-GPU batch workloads compared to CPU-GPU workloads. We further demonstrate that while specialized interconnects, NVLink, can improve batch cost, their benefits are limited by host OS software overhead and GPU oversubscription. This study serves as a proxy for future shared memory systems, such as those that interface with HMM, and the development of interconnects.

Study on Heterogeneous Queuing

Task Scheduling of Data-Parallel Applications on HSA Platform.

QoS Management on Heterogeneous Architecture for Multiprogrammed, Parallel, and Domain-Specific Applications

An Adaptive Algorithm of K-Means on Hsa Platform

HyQ: Hybrid I/O Queue Architecture for NVMe over Fabrics to Enable High- Performance Hardware Offloading

SQUASH: Simple QoS-Aware High-Performance Memory Scheduler for Heterogeneous Systems with Hardware Accelerators

Simultaneous and Heterogenous Multithreading: Exploiting Simultaneous and Heterogeneous Parallelism in Accelerator-Rich Architectures

Improving the Efficiency of GPGPU Work-Queue Through Data Awareness.

Review Of Heterogeneous Computing Based On Gpu

Communication and Memory Access Latency Characteristics of CPU/GPU Heterogeneous Cluster

Exploration on Task Scheduling Strategy for CPU-GPU Heterogeneous Computing System

A CPU-GPGPU Scheduler Based on Data Transmission Bandwidth of Workload

Performance Optimization for CPU-GPU Heterogeneous Parallel System

Improving the Performance of Heterogeneous Multi-Core Processors by Modifying the Cache Coherence Protocol

An approach to accessing unified memory address space of heterogeneous kilo-cores system

Fine-Grain Quantitative Analysis of Demand Paging in Unified Virtual Memory

A user mode CPU–GPU scheduling framework for hybrid workloads

A Survey on Heterogeneous CPU–GPU Architectures and Simulators

Processes Scheduling on Heterogeneous Multi-core Architecture with Hardware Support.

Resource Scheduling Strategy for Performance Optimization Based on Heterogeneous CPU-GPU Platform

Applications Classification and Scheduling on Heterogeneous HPC Systems Using Experimental Research.