Abstract:Multiphase compressible flows are often characterized by a broad range of space and time scales, entailing large grids and small time steps. Simulations of these flows on CPU-based clusters can thus take several wall-clock days. Offloading the compute kernels to GPUs appears attractive but is memory-bound for many finite-volume and -difference methods, damping speedups. Even when realized, GPU-based kernels lead to more intrusive communication and I/O times owing to lower computation costs. We present a strategy for GPU acceleration of multiphase compressible flow solvers that addresses these challenges and obtains large speedups at scale. We use OpenACC for directive-based offloading of all compute kernels while maintaining low-level control when needed. An established Fortran preprocessor and metaprogramming tool, Fypp, enables otherwise hidden compile-time optimizations. This strategy exposes compile-time optimizations and high memory reuse while retaining readable, maintainable, and compact code. Remote direct memory access realized via CUDA-aware MPI and GPUDirect reduces halo-exchange communication time. We implement this approach in the open-source solver MFC [1] . Metaprogramming results in an 8-times speedup of the most expensive kernels compared to a statically compiled program, reaching 46% of peak FLOPs on modern NVIDIA GPUs and high arithmetic intensity (about 10 FLOPs/byte). In representative simulations, a single NVIDIA A100 GPU is 7-times faster compared to an Intel Xeon Cascade Lake (6248) CPU die, or about 300-times faster compared to a single such CPU core. At the same time, near-ideal (97%) weak scaling is observed for at least 13824 GPUs on OLCF Summit. A strong scaling efficiency of 84% is retained for an 8-times increase in GPU count. Collective I/O, implemented via MPI3, helps ensure the negligible contribution of data transfers ( <1% of the wall time for a typical, large simulation). Large many-GPU simulations of compressible (solid-)liquid-gas flows demonstrate the practical utility of this strategy.

Using the Discrete Wavelet Transform for Lossy On‐the‐Fly Compression of GPU Fluid Simulations

Reducing the memory usage of Lattice-Boltzmann schemes with a DWT-based compression

A generic parallel computational framework of lifting wavelet transform for online engineering surface filtration.

Efficient odd–even multigrid for pointwise incompressible fluid simulation on GPU

Method for scalable and performant GPU-accelerated simulation of multiphase compressible flow

Pass-efficient methods for compression of high-dimensional turbulent flow data

Method for portable, scalable, and performant GPU-accelerated simulation of multiphase compressible flow

Wavelet Potentials: An Efficient Potential Recovery Technique for Pointwise Incompressible Fluids

Performance engineering for the Lattice Boltzmann method on GPGPUs: Architectural requirements and performance results

Real-Time Incompressible Fluid Simulation on the GPU.

Three-Dimensional Numerical Method for Simulating Large-Scale Free Water Surface by Massive Parallel Computing on a GPU

Reducing memory requirements for large size LBM simulations on GPUs

Fast Compressed Segmentation Volumes for Scientific Visualization

Massively Distributed Finite-Volume Flux Computation

A Coupled Grid-Particle Method for Fluid Animation on GPU

Real-time 2-D Fluid Simulation with a GPU Acceleration

An Improved Study of Physically Based Fluid Simulation on GPU

GPU Optimization for High-Quality Kinetic Fluid Simulation

Watertight Incremental Heightfield Tessellation

Hydrodynamic Simulations using GPGPU Architectures

FZ-GPU: A Fast and High-Ratio Lossy Compressor for Scientific Computing Applications on GPUs