Abstract:Positive semi-definite matrices commonly occur as normal matrices of least squares problems in statistics or as kernel matrices in machine learning and approximation theory. They are typically large and dense. Thus algorithms to solve systems with such a matrix can be very costly. A core idea to reduce computational complexity is to approximate the matrix by one with a low rank. The optimal and well understood choice is based on the eigenvalue decomposition of the matrix. Unfortunately, this is computationally very expensive. Cheaper methods are based on Gaussian elimination but they require pivoting. We show how invariant matrix theory provides explicit error formulas for an averaged error based on volume sampling. The formula leads to ratios of elementary symmetric polynomials on the eigenvalues. We discuss several bounds for the expected norm of the approximation error and include examples where this expected error norm can be computed exactly. References A. Dax. “On extremum properties of orthogonal quotients matrices”. In: Lin. Alg. Appl. 432.5 (2010), pp. 1234–1257. doi: 10.1016/j.laa.2009.10.034. M. Dereziński and M. W. Mahoney. Determinantal Point Processes in Randomized Numerical Linear Algebra. 2020. url: https://arxiv.org/abs/2005.03185. A. Deshpande, L. Rademacher, S. Vempala, and G. Wang. “Matrix approximation and projective clustering via volume sampling”. In: Proceedings of the Seventeenth Annual ACM-SIAM Symposium on Discrete Algorithm. SODA ’06. Miami, Florida: Society for Industrial and Applied Mathematics, 2006, pp. 1117–1126. url: https://dl.acm.org/doi/abs/10.5555/1109557.1109681. S. A. Goreinov, E. E. Tyrtyshnikov, and N. L. Zamarashkin. “A theory of pseudoskeleton approximations”. In: Lin. Alg. Appl. 261.1 (1997), pp. 1–21. doi: 10.1016/S0024-3795(96)00301-1. M. W. Mahoney and P. Drineas. “CUR matrix decompositions for improved data analysis”. In: Proc. Nat. Acad. Sci. 106.3 (Jan. 20, 2009), pp. 697–702. doi: 10.1073/pnas.0803205106. M. Marcus and L. Lopes. “Inequalities for symmetric functions and Hermitian matrices”. In: Can. J. Math. 9 (1957), pp. 305–312. doi: 10.4153/CJM-1957-037-9.

Multiplicative iterative algorithm for the low rank approximation of the symmetric positive semi-definite matrix

An Efficient Gauss--Newton Algorithm for Symmetric Low-Rank Product Matrix Approximations

A Symmetric Rank-one Quasi Newton Method for Non-negative Matrix Factorization

Multiplicative updates for symmetric-cone factorizations

The Anti-Symmetric Ortho-Symmetric Solution of A Linear Matrix Equation and Its Optimal Approximation

An algorithm for low-rank matrix factorization and its applications

Adaptive iterative singular value thresholding algorithm to low-rank matrix recovery

Preconditioned Low-Rank Riemannian Optimization for Symmetric Positive Definite Linear Matrix Equations

Efficient Method for Symmetric Nonnegative Matrix Factorization with an Approximate Augmented Lagrangian Scheme

An efficient numerical algorithm for solving linear systems with cyclic tridiagonal coefficient matrices

A Fast Algorithm for Nonnegative Matrix Factorization and Its Convergence.

A fast Multiplicative Updates algorithm for Non-negative Matrix Factorization

An implicit iterative algorithm for generalized algebraic Riccati matrix equations

Approximation Conjugate Gradient Method for Low-Rank Matrix Recovery

A generalized Nystrom method with column sketching for low-rank approximation of nonsymmetric matrices

An Implicit Iterative Algorithm for Algebraic Riccati Matrix Equations

A Two-Phase Algorithm for Robust Symmetric Non-Negative Matrix Factorization

Efficient Randomized Algorithms for the Fixed-Precision Low-Rank Matrix Approximation

Low rank approximation of positive semi-definite symmetric matrices using Gaussian elimination and volume sampling

The Application of ACA and SVD Algorithms in the Matrix Low-Rank Approximation

A Parallel Matrix Multiplication Algorithm for Some Cauchy-Like Matrices