Abstract:Positive semi-definite matrices commonly occur as normal matrices of least squares problems in statistics or as kernel matrices in machine learning and approximation theory. They are typically large and dense. Thus algorithms to solve systems with such a matrix can be very costly. A core idea to reduce computational complexity is to approximate the matrix by one with a low rank. The optimal and well understood choice is based on the eigenvalue decomposition of the matrix. Unfortunately, this is computationally very expensive. Cheaper methods are based on Gaussian elimination but they require pivoting. We show how invariant matrix theory provides explicit error formulas for an averaged error based on volume sampling. The formula leads to ratios of elementary symmetric polynomials on the eigenvalues. We discuss several bounds for the expected norm of the approximation error and include examples where this expected error norm can be computed exactly. References A. Dax. “On extremum properties of orthogonal quotients matrices”. In: Lin. Alg. Appl. 432.5 (2010), pp. 1234–1257. doi: 10.1016/j.laa.2009.10.034. M. Dereziński and M. W. Mahoney. Determinantal Point Processes in Randomized Numerical Linear Algebra. 2020. url: https://arxiv.org/abs/2005.03185. A. Deshpande, L. Rademacher, S. Vempala, and G. Wang. “Matrix approximation and projective clustering via volume sampling”. In: Proceedings of the Seventeenth Annual ACM-SIAM Symposium on Discrete Algorithm. SODA ’06. Miami, Florida: Society for Industrial and Applied Mathematics, 2006, pp. 1117–1126. url: https://dl.acm.org/doi/abs/10.5555/1109557.1109681. S. A. Goreinov, E. E. Tyrtyshnikov, and N. L. Zamarashkin. “A theory of pseudoskeleton approximations”. In: Lin. Alg. Appl. 261.1 (1997), pp. 1–21. doi: 10.1016/S0024-3795(96)00301-1. M. W. Mahoney and P. Drineas. “CUR matrix decompositions for improved data analysis”. In: Proc. Nat. Acad. Sci. 106.3 (Jan. 20, 2009), pp. 697–702. doi: 10.1073/pnas.0803205106. M. Marcus and L. Lopes. “Inequalities for symmetric functions and Hermitian matrices”. In: Can. J. Math. 9 (1957), pp. 305–312. doi: 10.4153/CJM-1957-037-9.

Symmetric matrix inversion using modified Gaussian elimination

Complex matrix inversion via real matrix inversions

A modification of the Jacobi-Davidson method

A Modified Inverse Iteration Method for Computing the Symmetric Tridiagonal Eigenvectors

Concurrent Gaussian elimination

Extended Newton-type Method for Inverse Singular Value Problems with Multiple And/or Zero Singular Values

Combinatorial and Recurrent Approaches for Efficient Matrix Inversion: Sub-cubic algorithms leveraging Fast Matrix products

Generalized Golub-Kahan bidiagonalization for nonsymmetric saddle point systems

Constraint Preconditioners for Symmetric Indefinite Matrices

Latest Inversion-Free Iterative Scheme for Solving a Pair of Nonlinear Matrix Equations

Low rank approximation of positive semi-definite symmetric matrices using Gaussian elimination and volume sampling

Gradient-type subspace iteration methods for the symmetric eigenvalue problem

Computing Symmetric Positive Definite Solutions of Three Types of Nonlinear Matrix Equations

A Relaxed Algorithm for Online Matrix Inversion

On Semi-Convergence and Inexact Iteration of the GSS Iteration Method for Nonsymmetric Singular Saddle Point Problems

SelInv---An Algorithm for Selected Inversion of a Sparse Symmetric Matrix

A Nesterov-style Accelerated Gradient Descent Algorithm for the Symmetric Eigenvalue Problem

Modified Newton'S Algorithm for Computing the Group Inverses of Singular Toeplitz Matrices

A generalized skew-symmetric Lanczos bidiagonalization method for computing several extreme eigenpairs of a large skew-symmetric/symmetric positive definite matrix pair

Fast Parallelizable Methods for Computing Invariant Subspaces of Hermitian Matrices

Integer Levinson Algorithm for the Inversion of any Nonsingular Hermitian Toeplitz Matrix