Abstract:We study theoretical properties of block solvers for the eigenvalue problem. Despite a recent surge of interest in such eigensolver analysis, truly block solvers have received relatively less attention, in contrast to the majority of studies concentrating on vector versions and non-truly block versions that rely on the deflation strategy. In fact, truly block solvers are more widely deployed in practice by virtue of its simplicity without compromise on accuracy. However, the corresponding theoretical analysis remains inadequate for first-order solvers, as only local and kth gap-dependent rates of convergence have been established thus far. This paper is devoted to revealing significantly better or asyet-unknown theoretical properties of such solvers. We present a novel convergence analysis in a unified framework for three types of first-order Riemannian solvers, i.e., deterministic, vanilla stochastic, and stochastic with variance reduction, that are to find top-k eigenvectors of a real symmetric matrix, in full generality. In particular, the issue of zero gaps between eigenvalues, to the best of our knowledge for the first time, is explicitly considered for these solvers, which brings new understandings, e.g., the dependence of convergence on gaps other than the k-th one. We thus propose the concept of generalized k-th gap. Three types of solvers are proved to converge to a globally optimal solution at a global, generalized k-th gap-dependent, and linear or sub-linear rate.

A Refined Block Conjugate Gradient Method for Symmetric Eigenvalue Problems

A Robust and Efficient Implementation of LOBPCG

A mixed precision LOBPCG algorithm

Generalized Block Lanczos Methods for Large Unsymmetric Eigenproblems

Practical Restrictively Preconditioned Conjugate Gradient Methods for a Class of Block Two-by-Two Linear Systems

Riemannian Linearized Proximal Algorithms for Nonnegative Inverse Eigenvalue Problem.

A projected preconditioned conjugate gradient algorithm for computing many extreme eigenpairs of a Hermitian matrix

A spectrally preconditioned and initially deflated variant of the restarted block GMRES method for solving multiple right-hand sides linear systems

A New Projected Variant of the Deflated Block Conjugate Gradient Method

Efficient Block Preconditioned Eigensolvers for Linear Response Time-Dependent Density Functional Theory.

A Two-Level Block Preconditioned Jacobi–Davidson Method for Multiple and Clustered Eigenvalues of Elliptic Operators

A Refined Iterative Algorithm Based on the Block Arnoldi Process for Large Unsymmetric Eigenproblems

Restrictive Preconditioners for Conjugate Gradient Methods for Symmetric Positive Definite Linear Systems

A Variation on the Block Arnoldi Method for Large Unsymmetric Matrix Eigenproblems

Block Algorithms with Augmented Rayleigh-Ritz Projections for Large-Scale Eigenpair Computation

A Refined Variant of the Inverse-Free Krylov Subspace Method for Symmetric Generalized Eigenvalue Problems

On Truly Block Eigensolvers Via Riemannian Optimization.

A Preconditioned MINRES Method for Block Lower Triangular Toeplitz Systems

A Refined Subspace Iteration Algorithm For Large Sparse Eigenproblems

Deflated Block Krylov Subspace Methods for Large Scale Eigenvalue Problems.