Abstract:The trend towards highly parallel multi-processing is ubiquitous in all modern computer architectures, ranging from handheld devices to large-scale HPC systems; yet many applications are struggling to fully utilise the multiple levels of parallelism exposed in modern high-performance platforms. In order to realise the full potential of recent hardware advances, a mixed-mode between shared-memory programming techniques and inter-node message passing can be adopted which provides high-levels of parallelism with minimal overheads. For scientific applications this entails that not only the simulation code itself, but the whole software stack needs to evolve. In this paper, we evaluate the mixed-mode performance of PETSc, a widely used scientific library for the scalable solution of partial differential equations. We describe the addition of OpenMP threaded functionality to the library, focusing on sparse matrix-vector multiplication. We highlight key challenges in achieving good parallel performance, such as explicit communication overlap using task-based parallelism, and show how to further improve performance by explicitly load balancing threads within MPI processes. Using a set of matrices extracted from Fluidity, a CFD application code which uses the library as its linear solver engine, we then benchmark the parallel performance of mixed-mode PETSc across multiple nodes on several modern HPC architectures. We evaluate the parallel scalability on Uniform Memory Access (UMA) systems, such as the Fujitsu PRIMEHPC FX10 and IBM BlueGene/Q, as well as a Non-Uniform Memory Access (NUMA) Cray XE6 platform. A detailed comparison is performed which highlights the characteristics of each particular architecture, before demonstrating efficient strong scalability of sparse matrix-vector multiplication with significant speedups over the pure-MPI mode.

Parallelization of Thermal Recovery Simulation Based on PETSc

A Parallel Simulator for Massive Reservoir Models Utilizing Distributed-Memory Parallel Systems

Parallel Computation for Reservoir Thermal Simulation of Multicomponent and Multiphase Fluid Flow

Reservoir Thermal Recover Simulation on Parallel Computers

THE IMPLEMENTATION OF A PARALLEL SOFTWARE OF PETROLEUM RESERVOIR SIMULATION AND ITS APPLICATION ON NATIVE HIGH PERFORMANCE COMPUTERS

Parallel Multiphysics Simulation for Characterizing Performance Degradation of RF Components

Reservoir Thermal Recovery Simulation Software Parallelization

An Efficient Method for Optimizing PETSc on the Sunway TaihuLight System.

Cost-Effective Parallel Reservoir Simulation on Shared Memory

Application of PETSc in Soil-Water Coupled Geotechnical Problems

Parallel Transient Response Topology Optimization Based on PETSc

Extending a serial 3D two-phase CFD code to parallel execution over MPI by using the PETSc library for domain decomposition

Field-split Preconditioned Active-Set Reduced-Space Algorithm for Complex Black Oil Reservoir Simulation at Large-Scale

Benchmarking mixed-mode PETSc performance on high-performance architectures

Valid Parallel Algorithm for Nonuniform Domain Simulation of Reservoir

Large-scale reservoir simulations on distributed-memory parallel computers.

A Scalable Parallel In-Situ Combustion Reservoir Simulator for Large Scale Models

On Robust and Efficient Parallel Reservoir Simulation on Tianhe 2

Parallel Computational Hydrodynamics Solver PNAOE-SJTU

A Parallel Framework for Reservoir Simulators on Distributed-memory Supercomputers

THC-MP: High Performance Numerical Simulation of Reactive Transport and Multiphase Flow in Porous Media.