Abstract:Positive semi-definite matrices commonly occur as normal matrices of least squares problems in statistics or as kernel matrices in machine learning and approximation theory. They are typically large and dense. Thus algorithms to solve systems with such a matrix can be very costly. A core idea to reduce computational complexity is to approximate the matrix by one with a low rank. The optimal and well understood choice is based on the eigenvalue decomposition of the matrix. Unfortunately, this is computationally very expensive. Cheaper methods are based on Gaussian elimination but they require pivoting. We show how invariant matrix theory provides explicit error formulas for an averaged error based on volume sampling. The formula leads to ratios of elementary symmetric polynomials on the eigenvalues. We discuss several bounds for the expected norm of the approximation error and include examples where this expected error norm can be computed exactly. References A. Dax. “On extremum properties of orthogonal quotients matrices”. In: Lin. Alg. Appl. 432.5 (2010), pp. 1234–1257. doi: 10.1016/j.laa.2009.10.034. M. Dereziński and M. W. Mahoney. Determinantal Point Processes in Randomized Numerical Linear Algebra. 2020. url: https://arxiv.org/abs/2005.03185. A. Deshpande, L. Rademacher, S. Vempala, and G. Wang. “Matrix approximation and projective clustering via volume sampling”. In: Proceedings of the Seventeenth Annual ACM-SIAM Symposium on Discrete Algorithm. SODA ’06. Miami, Florida: Society for Industrial and Applied Mathematics, 2006, pp. 1117–1126. url: https://dl.acm.org/doi/abs/10.5555/1109557.1109681. S. A. Goreinov, E. E. Tyrtyshnikov, and N. L. Zamarashkin. “A theory of pseudoskeleton approximations”. In: Lin. Alg. Appl. 261.1 (1997), pp. 1–21. doi: 10.1016/S0024-3795(96)00301-1. M. W. Mahoney and P. Drineas. “CUR matrix decompositions for improved data analysis”. In: Proc. Nat. Acad. Sci. 106.3 (Jan. 20, 2009), pp. 697–702. doi: 10.1073/pnas.0803205106. M. Marcus and L. Lopes. “Inequalities for symmetric functions and Hermitian matrices”. In: Can. J. Math. 9 (1957), pp. 305–312. doi: 10.4153/CJM-1957-037-9.

Accurate singular values of a class of parameterized negative matrices

Accurate computations of singular values and linear systems for Polynomial-Vandermonde-type matrices

Extended Newton-type Method for Inverse Singular Value Problems with Multiple And/or Zero Singular Values

Uniform Approximation of Eigenproblems of a Large-Scale Parameter-Dependent Hermitian Matrix

An Extended Ulm-like Method for Inverse Singular Value Problems with Multiple And/or Zero Singular Values

Newton-type Methods for Inverse Singular Value Problems with Multiple Singular Values

Convergence of a Ulm-like Method for Square Inverse Singular Value Problems with Multiple and Zero Singular Values.

Convergence of the Newton-type methods for the square inverse singular value problems with multiple and zero singular values

An Inexact Newton-type Method for Inverse Singular Value Problems

Low rank approximation of positive semi-definite symmetric matrices using Gaussian elimination and volume sampling

Adaptive iterative singular value thresholding algorithm to low-rank matrix recovery

How Over-Parameterization Slows Down Gradient Descent in Matrix Sensing: The Curses of Symmetry and Initialization

Nonnegative Low Rank Matrix Approximation for Nonnegative Matrices

High relative accuracy for rational q-Bernstein–Vandermonde matrices

Detecting negative eigenvalues of exact and approximate Hessian matrices in optimization

A calibration method for non-positive definite covariance matrix in multivariate data analysis

A new lower bound for the smallest singular value

A generalized Nystrom method with column sketching for low-rank approximation of nonsymmetric matrices

Matrix perturbation analysis of methods for extracting singular values from approximate singular subspaces

Approximation Accuracy of the Krylov Subspaces for Linear Discrete Ill-Posed Problems