Abstract:Current supercomputers often have a heterogeneous architecture using both CPUs and GPUs. At the same time, numerical simulation tasks frequently involve multiphysics scenarios whose components run on different hardware due to multiple reasons, e.g., architectural requirements, pragmatism, etc. This leads naturally to a software design where different simulation modules are mapped to different subsystems of the heterogeneous architecture. We present a detailed performance analysis for such a hybrid four-way coupled simulation of a fully resolved particle-laden flow. The Eulerian representation of the flow utilizes GPUs, while the Lagrangian model for the particles runs on CPUs. First, a roofline model is employed to predict the node level performance and to show that the lattice-Boltzmann-based fluid simulation reaches very good performance on a single GPU. Furthermore, the GPU-GPU communication for a large-scale flow simulation results in only moderate slowdowns due to the efficiency of the CUDA-aware MPI communication, combined with communication hiding techniques. On 1024 A100 GPUs, a parallel efficiency of up to 71% is achieved. While the flow simulation has good performance characteristics, the integration of the stiff Lagrangian particle system requires frequent CPU-CPU communications that can become a bottleneck. Additionally, special attention is paid to the CPU-GPU communication overhead since this is essential for coupling the particles to the flow simulation. However, thanks to our problem-aware co-partitioning, the CPU-GPU communication overhead is found to be negligible. As a lesson learned from this development, four criteria are postulated that a hybrid implementation must meet for the efficient use of heterogeneous supercomputers. Additionally, an a priori estimate of the speedup for hybrid implementations is suggested.

An Alternative Approach for Collaborative Simulation Execution on a CPU+GPU Hybrid System

The Implementation of the Three-Dimensional Unified Gas-Kinetic Wave-Particle Method on Multiple Graphics Processing Units

Parallelized Implementation of the Finite Particle Method for Explicit Dynamics in GPU

Gpu-Assisted Hybrid Network Traffic Model

Exploiting Parallelism in the Simulation of General Purpose Graphics Processing Unit Program

Parallel Transient Stability-Constrained Optimal Power Flow Using GPU as Coprocessor.

A Preliminary Study on Accelerating Simulation Optimization with GPU Implementation

CPU-GPU Heterogeneous Code Acceleration of a Finite Volume Computational Fluid Dynamics Solver

Heterogeneous computing in a strongly-connected CPU-GPU environment: fast multiple time-evolution equation-based modeling accelerated using data-driven approach

Gpu-Accelerated Evaluation Platform For High Fidelity Network Modeling

A GPU Based Parallel Computing Mechanism of an Etching Profile Evolution Model.

Unleashing the Performance Potential of CPU-GPU Platforms for the 3D Atmospheric Euler Solver.

Efficiency and scalability of fully-resolved fluid-particle simulations on heterogeneous CPU-GPU architectures

Accelerating the 3D Euler Atmospheric Solver Through Heterogeneous CPU-GPU Platforms

Orchestrated Co-scheduling, Resource Partitioning, and Power Capping on CPU-GPU Heterogeneous Systems via Machine Learning

A hybrid CPU‐GPU paradigm to accelerate reactive CFD simulations

A GPU parallel scheme for accelerating 2D and 3D peridynamics models

CAP: co-scheduling based on asymptotic profiling in CPU+GPU hybrid systems.

Accelerating CFD simulation with high order finite difference method on curvilinear coordinates for modern GPU clusters

A Scalable Hybrid Algorithm for Solving Partial Differential Equations on a Cluster of CPU/GPU

Comparison of Parallel Implementations of Controls on Gpu for Transient Simulation of Power System