Abstract:Interest in communication-avoiding orthogonalization schemes for high-performance computing has been growing recently. This manuscript addresses open questions about the numerical stability of various block classical Gram-Schmidt variants that have been proposed in the past few years. An abstract framework is employed, the flexibility of which allows for new rigorous bounds on the loss of orthogonality in these variants. We first analyze a generalization of (reorthogonalized) block classical Gram-Schmidt and show that a "strong" intrablock orthogonalization routine is only needed for the very first block in order to maintain orthogonality on the level of the unit roundoff. Then, using this variant, which has four synchronization points per block column, we remove the synchronization points one at a time and analyze how each alteration affects the stability of the resulting method. Our analysis shows that the variant requiring only one synchronization per block column cannot be guaranteed to be stable in practice, as stability begins to degrade with the first reduction of synchronization points. Our analysis of block methods also provides new theoretical results for the single-column case. In particular, it is proven that DCGS2 from [Bielich, D. et al. Par. Comput. 112 (2022)] and CGS-2 from [Świrydowicz, K. et al, Num. Lin. Alg. Appl. 28 (2021)] are as stable as Householder QR. Numerical examples from the BlockStab toolbox are included throughout, to help compare variants and illustrate the effects of different choices of intraorthogonalization subroutines.

Analyzing Vector Orthogonalization Algorithms

A Kaczmarz-Inspired Method for Orthogonalization

Optimal Randomized First-Order Methods for Least-Squares Problems

Computationally efficient orthogonalization for pairwise comparisons method

On Construction of a Class of Orthogonal Arrays (Thesis)

Householder Orthogonalization with a Non-Standard Inner Product

Approximate Solutions of Linear Systems at a Universal Rate

Approximation of Gram-Schmidt Orthogonalization by Data Matrix

Matrix orthogonal polynomials whose derivatives are also orthogonal

Approximation of Bounds on Mixed Level Orthogonal Arrays

Backward Error Analysis of the Lanczos Bidiagonalization with Reorthogonalization.

Orthogonal orthogonal overcomplete kernel design for sparse representation

On the loss of orthogonality in low-synchronization variants of reorthogonalized block classical Gram-Schmidt

Infeasible Deterministic, Stochastic, and Variance-Reduction Algorithms for Optimization under Orthogonality Constraints

A general error analysis for randomized low-rank approximation with application to data assimilation

Generation of Random (Generalized) Orthogonal Matrices

On The Tensor Svd And The Optimal Low Rank Orthogonal Approximation Of Tensors

Composite orthogonal projection methods for large matrix eigenproblems

Explicit Orthogonal Arrays and Universal Hashing with Arbitrary Parameters

Almost Orthogonal Arrays: Theory and Search Three Ways

A Splitting Method for Orthogonality Constrained Problems