Abstract:Positive semi-definite matrices commonly occur as normal matrices of least squares problems in statistics or as kernel matrices in machine learning and approximation theory. They are typically large and dense. Thus algorithms to solve systems with such a matrix can be very costly. A core idea to reduce computational complexity is to approximate the matrix by one with a low rank. The optimal and well understood choice is based on the eigenvalue decomposition of the matrix. Unfortunately, this is computationally very expensive. Cheaper methods are based on Gaussian elimination but they require pivoting. We show how invariant matrix theory provides explicit error formulas for an averaged error based on volume sampling. The formula leads to ratios of elementary symmetric polynomials on the eigenvalues. We discuss several bounds for the expected norm of the approximation error and include examples where this expected error norm can be computed exactly. References A. Dax. “On extremum properties of orthogonal quotients matrices”. In: Lin. Alg. Appl. 432.5 (2010), pp. 1234–1257. doi: 10.1016/j.laa.2009.10.034. M. Dereziński and M. W. Mahoney. Determinantal Point Processes in Randomized Numerical Linear Algebra. 2020. url: https://arxiv.org/abs/2005.03185. A. Deshpande, L. Rademacher, S. Vempala, and G. Wang. “Matrix approximation and projective clustering via volume sampling”. In: Proceedings of the Seventeenth Annual ACM-SIAM Symposium on Discrete Algorithm. SODA ’06. Miami, Florida: Society for Industrial and Applied Mathematics, 2006, pp. 1117–1126. url: https://dl.acm.org/doi/abs/10.5555/1109557.1109681. S. A. Goreinov, E. E. Tyrtyshnikov, and N. L. Zamarashkin. “A theory of pseudoskeleton approximations”. In: Lin. Alg. Appl. 261.1 (1997), pp. 1–21. doi: 10.1016/S0024-3795(96)00301-1. M. W. Mahoney and P. Drineas. “CUR matrix decompositions for improved data analysis”. In: Proc. Nat. Acad. Sci. 106.3 (Jan. 20, 2009), pp. 697–702. doi: 10.1073/pnas.0803205106. M. Marcus and L. Lopes. “Inequalities for symmetric functions and Hermitian matrices”. In: Can. J. Math. 9 (1957), pp. 305–312. doi: 10.4153/CJM-1957-037-9.

To solving the eigenvalue problem for polynomial matrices of general form

Fast and backward stable computation of the eigenvalues and eigenvectors of matrix polynomials

Numerical methods for eigenvalues of singular polynomial eigenvalue problems

On the application of Laguerre's method to the polynomial eigenvalue problem

Minimal rank factorizations of polynomial matrices

Eigenvalue bounds for matrix polynomials in generalized bases

Low-Rank Matrix Iteration Using Polynomial-Filtered Subspace Extraction.

Exact Algorithms for Computing Generalized Eigenspaces of Matrices via Annihilating Polynomials

Eigenvalue programming beyond matrices

Inflationary dynamics for matrix eigenvalue problems

Eigen-solving Via Reduction to DPR1 Matrices

Fast and Accurate Randomized Algorithms for Linear Systems and Eigenvalue Problems

Root-Finding with Eigen-Solving

The shift-and-invert Arnoldi method for singular matrix pencils

An Explicit Solution to Polynomial Matrix Right Coprime Factorization with Application in Eigenstructure Assignment

A Variation on the Block Arnoldi Method for Large Unsymmetric Matrix Eigenproblems

On a General Solution of Partially Described Inverse Quadratic Eigenvalue Problems and Its Applications

Low rank approximation of positive semi-definite symmetric matrices using Gaussian elimination and volume sampling

Average polynomial time for eigenvector computations

Generalization and variations of Pellet's theorem for matrix polynomials

Analysis of eigenvalue condition numbers for a class of randomized numerical methods for singular matrix pencils