Abstract:Krylov methods provide a fast and highly parallel numerical tool for the iterative solution of many large-scale sparse linear systems. To a large extent, the performance of practical realizations of these methods is constrained by the communication bandwidth in all current computer architectures, motivating the recent investigation of sophisticated techniques to avoid, reduce, and/or hide the message-passing costs (in distributed platforms) and the memory accesses (in all architectures). This paper introduces a new communication-reduction strategy for the (Krylov) GMRES solver that advocates for decoupling the storage format (i.e., the data representation in memory) of the orthogonal basis from the arithmetic precision that is employed during the operations with that basis. Given that the execution time of the GMRES solver is largely determined by the memory access, the datatype transforms can be mostly hidden, resulting in the acceleration of the iterative step via a lower volume of bits being retrieved from memory. Together with the special properties of the orthonormal basis (whose elements are all bounded by 1), this paves the road toward the aggressive customization of the storage format, which includes some floating point as well as fixed point formats with little impact on the convergence of the iterative process. We develop a high performance implementation of the "compressed basis GMRES" solver in the Ginkgo sparse linear algebra library and using a large set of test problems from the SuiteSparse matrix collection we demonstrate robustness and performance advantages on a modern NVIDIA V100 GPU of up to 50% over the standard GMRES solver that stores all data in IEEE double precision.

Implicitly restarted and deflated GMRES

Randomized flexible GMRES with deflated restarting

Block GMRES Method with Inexact Breakdowns and Deflated Restarting

GMRES with randomized sketching and deflated restarting

Randomized Implicitly Restarted Arnoldi method for the non-symmetric eigenvalue problem

Deflated Iterative Methods for Linear Equations with Multiple Right-Hand Sides

Improving eigenvectors in Arnoldi's method

Preconditioning Low Rank Generalized Minimal Residual Method (GMRES) for Implicit Discretizations of Matrix Differential Equations

On IGMRES: an Incomplete Generalized Minimal Residual Method for Large Unsymmetric Linear Systems

Deflation of eigenvalues for iterative methods in lattice QCD

A dynamically deflated GMRES adjoint solver for aerodynamic shape optimization

A Nonlinear GMRES Optimization Algorithm for Canonical Tensor Decomposition

Deflation of Eigenvalues for GMRES in Lattice QCD

Compressed Basis GMRES on High Performance GPUs

On the backward stability of s-step GMRES

Optimal Solutions of Well-Posed Linear Systems via Low-Precision Right-Preconditioned GMRES with Forward and Backward Stabilization

A restarted GMRES-based implementation of IDR(s)stab(L) to yield higher robustness

Deflated GMRES for systems with multiple shifts and multiple right-hand sides

A Proportional-Derivative Control Strategy for Restarting the GMRES(m) Algorithm.

Structure Preserving Restarts of the Non-Symmetric Lanczos Algorithm via the Implicitly shifted LR algorithm

A Refined Second-order Arnoldi (RSOAR) Method for the Quadratic Eigenvalue Problem and Implicit Restarting