Abstract:The abstraction of a shared memory space over separate CPU and GPU memory domains has eased the burden of portability for many HPC codebases. However, users pay for ease of use provided by system-managed memory with a moderate-to-high performance overhead. NVIDIA Unified Virtual Memory (UVM) is currently the primary real-world implementation of such abstraction and offers a functionally equivalent testbed for in-depth performance study for both UVM and future Linux Heterogeneous Memory Management (HMM) compatible systems. The continued advocacy for UVM and HMM motivates improvement of the underlying system. We focus on UVM-based systems and investigate root causes of UVM overhead, a non-trivial task due to complex interactions of multiple hardware and software constituents and the desired cost granularity. In our prior work, we delved deeply into UVM system architecture and showed internal behaviors of page fault servicing in batches. We provided quantitative evaluation of batch handling for various applications under different scenarios, including prefetching and oversubscription. We revealed the driver workload depends on the interactions among application access patterns, GPU hardware constraints, and host OS components. Host OS components have significant overhead present across implementations, warranting close attention. This extension furthers our prior study in three aspects: fine-grain cost analysis and breakdown, extension to multiple GPUs, and investigation of platforms with different GPU-GPU interconnects. We take a top-down approach to quantitative batch analysis and uncover how constituent component costs accumulate and overlap governed by synchronous and asynchronous operations. Our multi-GPU analysis shows reduced cost of GPU-GPU batch workloads compared to CPU-GPU workloads. We further demonstrate that while specialized interconnects, NVLink, can improve batch cost, their benefits are limited by host OS software overhead and GPU oversubscription. This study serves as a proxy for future shared memory systems, such as those that interface with HMM, and the development of interconnects.

GVARP: Detecting Performance Variance on Large-Scale Heterogeneous Systems

Detecting Performance Variance for Parallel Applications Without Source Code

VAPRO: Performance Variance Detection and Diagnosis for Production-Run Parallel Applications

A parallel computing method for irregular work

Vs Ensor

vSensor: leveraging fixed-workload snippets of programs for performance variance detection.

Varcatcher: A Framework for Tackling Performance Variability of Parallel Workloads on Multi-Core

Lightweight Noise Detection

Hybrid Performance Modeling And Analyzing Of Parallel Systems

A new tool for the performance analysis of massively parallel computer systems

Leveraging Code Snippets to Detect Variations in the Performance of HPC Systems

A parallel workload has extreme variability in a production environment

Communication Analysis And Performance Prediction Of Parallel Applications On Large-Scale Machines

Design Strategies and Approximation Methods for High-Performance Computing Variability Management

vPALs: Towards Verified Performance-aware Learning System For Resource Management

Fine-Grain Quantitative Analysis of Demand Paging in Unified Virtual Memory

Performance Visualization For Large-Scale Computing Systems: A Literature Review

Performance modeling of graphics processing unit application using static and dynamic analysis

HPC System Software Enhanced by Source Code Analysis

Graph-Centric Performance Analysis for Large-Scale Parallel Applications

Identifying Scalability Bottlenecks for Large-Scale Parallel Programs with Graph Analysis