Towards a Mini-App for Smoothed Particle Hydrodynamics at Exascale

Danilo Guerrera,Rubén M. Cabezón,Jean-Guillaume Piccinali,Aurélien Cavelan,Florina M. Ciorba,David Imbert,Lucio Mayer,Darren Reed

DOI: https://doi.org/10.1109/CLUSTER.2018.00077

2018-09-21

Abstract:The smoothed particle hydrodynamics (SPH) technique is a purely Lagrangian method, used in numerical simulations of fluids in astrophysics and computational fluid dynamics, among many other fields. SPH simulations with detailed physics represent computationally-demanding calculations. The parallelization of SPH codes is not trivial due to the absence of a structured grid. Additionally, the performance of the SPH codes can be, in general, adversely impacted by several factors, such as multiple time-stepping, long-range interactions, and/or boundary conditions. This work presents insights into the current performance and functionalities of three SPH codes: SPHYNX, ChaNGa, and SPH-flow. These codes are the starting point of an interdisciplinary co-design project, SPH-EXA, for the development of an Exascale-ready SPH mini-app. To gain such insights, a rotating square patch test was implemented as a common test simulation for the three SPH codes and analyzed on two modern HPC systems. Furthermore, to stress the differences with the codes stemming from the astrophysics community (SPHYNX and ChaNGa), an additional test case, the Evrard collapse, has also been carried out. This work extrapolates the common basic SPH features in the three codes for the purpose of consolidating them into a pure-SPH, Exascale-ready, optimized, mini-app. Moreover, the outcome of this serves as direct feedback to the parent codes, to improve their performance and overall scalability.

Computational Physics,Computational Engineering, Finance, and Science,Distributed, Parallel, and Cluster Computing

What problem does this paper attempt to address?

### Problems the paper attempts to solve

The paper aims to solve the following problems:

1. **Improve the performance and scalability of Smoothed Particle Hydrodynamics (SPH) code on Exascale computing platforms**:
- SPH is a mesh - free, purely Lagrangian method, widely used for fluid simulations in astrophysics and Computational Fluid Dynamics (CFD). However, the parallelization of SPH code is not straightforward because of the lack of a structured grid, which causes the interactions between fluid elements to change continuously over time.
- By analyzing the performance and functionality of three existing SPH codes (SPHYNX, ChaNGa, and SPH - flow), the paper reveals the performance and limitations of these codes on current High - Performance Computing (HPC) systems.

2. **Develop an Exascale - ready SPH mini - app**:
- This mini - app will integrate the best features of existing SPH codes and optimize its performance at multiple parallel levels.
- By creating a smaller, more manageable and easier - to - use mini - app, researchers can better study and optimize the performance of SPH algorithms on future Exascale computing platforms.

3. **Provide direct feedback on existing SPH codes**:
- Through in - depth analysis of existing SPH codes, the paper provides direct feedback on the performance improvement and overall scalability of these codes, which helps to improve the efficiency of these codes on existing and future Exascale platforms.

### Specific problems and solutions

- **Parallelization challenges**:
- **Problem**: The parallelization of SPH code is complicated due to the lack of a structured grid, resulting in continuous changes in the interactions between fluid elements over time.
- **Solution**: By analyzing the parallelization strategies of different SPH codes, the paper proposes Dynamic Load Balancing (DLB) techniques and other optimization methods to improve parallel efficiency.

- **Performance bottlenecks**:
- **Problem**: Existing SPH codes have performance bottlenecks when dealing with large - scale data and complex physical processes.
- **Solution**: By implementing multi - time - stepping, optimizing self - gravity calculations, etc., the paper reduces the computing time and improves performance.

- **Scalability**:
- **Problem**: Existing SPH codes are difficult to maintain good scalability on large - scale computing nodes.
- **Solution**: The paper proposes using domain decomposition methods such as Orthogonal Recursive Bisection and Space Filling Curves, as well as fault - tolerance mechanisms such as multilevel checkpointing, to improve the scalability of the code.

### Conclusion

Through the above - mentioned analysis and optimization, the paper lays the foundation for developing an efficient and scalable Exascale - ready SPH mini - app, and also provides a valuable reference for the performance improvement of existing SPH codes.