Abstract:GPGPU architectures have become established as the dominant parallelization and performance platform achieving exceptional popularization and empowering domains such as regular algebra, machine learning, image detection and self-driving cars. However, irregular applications struggle to fully realize GPGPU performance as a result of control flow divergence and memory divergence due to irregular memory access patterns. To ameliorate these issues, programmers are obligated to carefully consider architecture features and devote significant efforts to modify the algorithms with complex optimization techniques, which shift programmers priorities yet struggle to quell the shortcomings. We show that in graph-based GPGPU irregular applications these inefficiencies prevail, yet we find that it is possible to relax the strict relationship between thread and data processed to empower new optimizations. Based on this key idea, we propose the Irregular accesses Reorder Unit (IRU), a novel hardware extension tightly integrated in the GPGPU pipeline. The IRU reorders data processed by the threads on irregular accesses which significantly improves memory coalescing, and allows increased performance and energy efficiency. Additionally, the IRU is capable of filtering and merging duplicated irregular access which further improves graph-based irregular applications. Programmers can easily utilize the IRU with a simple API, or compiler optimized generated code with the extended ISA instructions provided. We evaluate our proposal for state-of-the-art graph-based algorithms and a wide selection of applications. Results show that the IRU achieves a memory coalescing improvement of 1.32x and a 46% reduction in the overall traffic in the memory hierarchy, which results in 1.33x and 13% improvement in performance and energy savings respectively, while incurring in a small 5.6% area overhead.

Cuirre: an Open-Source Library for Load Balancing and Characterizing Irregular Applications on Gpus

A parallel computing method for irregular work

Irregular Accesses Reorder Unit: Improving GPGPU Memory Coalescing for Graph-Based Workloads

FinePar: Irregularity-aware Fine-Grained Workload Partitioning on Integrated Architectures

Optimizing Non-Coalesced Memory Access for Irregular Applications with GPU Computing

A Programming Model for GPU Load Balancing

Just-In-Time Locality and Percolation for Optimizing Irregular Applications on A Manycore Architecture

Workload Analysis for Typical GPU Programs Using CUPTI Interface

Design of load balancing algorithm for CPU and GPU collaborative computing based on C4.5 decision tree.

Automatic Parallelization and Optimization for Irregular Scientific Applications.

An Adaptive Load Balancer For Graph Analytical Applications on GPUs

CPU-assisted GPU thread pool model for dynamic task parallelism

Parallelization Design of Irregular Algorithms of Video Processing on GPUs.

A Credit-Based Load-Balance-Aware CTA Scheduling Optimization Scheme in GPGPU

A tunable coarse-grained parallel algorithm for irregular dynamic programming applications

Performance optimization and modeling of fine-grained irregular communication in UPC

A Self-Adaptive Workload Balancing Algorithm on Gpu Clusters

cuFINUFFT: a load-balanced GPU library for general-purpose nonuniform FFTs

Improving Performance of GPU Specific OpenCL Program on CPUs

Toward QoS-Awareness and Improved Utilization of Spatial Multitasking GPUs

CuHMMer: A load-balanced CPU-GPU cooperative bioinformatics application