Abstract:The generalized Dryja--Smith--Widlund (GDSW) preconditioner is a two-level overlapping Schwarz domain decomposition (DD) preconditioner that couples a classical one-level overlapping Schwarz preconditioner with an energy-minimizing coarse space. When used to accelerate the convergence rate of Krylov subspace iterative methods, the GDSW preconditioner provides robustness and scalability for the solution of sparse linear systems arising from the discretization of a wide range of partial different equations. In this paper, we present FROSch (Fast and Robust Schwarz), a domain decomposition solver package which implements GDSW-type preconditioners for both CPU and GPU clusters. To improve the solver performance on GPUs, we use a novel decomposition to run multiple MPI processes on each GPU, reducing both solver's computational and storage costs and potentially improving the convergence rate. This allowed us to obtain competitive or faster performance using GPUs compared to using CPUs alone. We demonstrate the performance of FROSch on the Summit supercomputer with NVIDIA V100 GPUs, where we used NVIDIA Multi-Process Service (MPS) to implement our decomposition strategy. The solver has a wide variety of algorithmic and implementation choices, which poses both opportunities and challenges for its GPU implementation. We conduct a thorough experimental study with different solver options including the exact or inexact solution of the local overlapping subdomain problems on a GPU. We also discuss the effect of using the iterative variant of the incomplete LU factorization and sparse-triangular solve as the approximate local solver, and using lower precision for computing the whole FROSch preconditioner. Overall, the solve time was reduced by factors of about $2\times$ using GPUs, while the GPU acceleration of the numerical setup time depend on the solver options and the local matrix sizes.

Parallel Incomplete Factorization Preconditioning of Block Tridiagonal Linear Systems with 2-D Domain Decomposition

Parallel incomplete factorization preconditioning of block tridiagonal linear systems with 2-D domain decomposition

Parallelization of incomplete factorization preconditioning of block tridiagonal linear systems with 1-D domain decomposition

A Parallelization Technique Based on Factor Combination and Graph Partitioning for General Incomplete LU Factorization

High Performance Block Incomplete LU Factorization

A Two-level GPU-Accelerated Incomplete LU Preconditioner for General Sparse Linear Systems

Parallel Factorizations in Numerical Analysis

Impact of two factors on several domain decomposition based parallel incomplete factorizations for the meso-scale simulation of concrete

Decomposition method for block-tridiagonal matrix systems

A Class of Inexact Block Factorization Preconditioners for Indefinite Matrices with a Three-by-three Block Structure.

Parallel Triangular Solvers on GPU

Incomplete Factorization Preconditioning for Linear Least Squares Problems

Parallel Preconditioners for Large Scale Partial Difference Equation Systems

Parallel Domain Decomposition Methods with Mixed Order Discretization for Fully Implicit Solution of Tracer Transport Problems on the Cubed-Sphere.

Block-circulant with circulant-block preconditioners for two-dimensional spatial fractional diffusion equations

An Experimental Study of Two-Level Schwarz Domain Decomposition Preconditioners on GPUs

Preconditioners for Parallel Reservoir Simulation.

Practical Restrictively Preconditioned Conjugate Gradient Methods for a Class of Block Two-by-Two Linear Systems

A Preconditioned MINRES Method for Block Lower Triangular Toeplitz Systems

Parallelization in time by diagonalization

Parallel Tiled QR Factorization for Multicore Architectures