Abstract:Positive semi-definite matrices commonly occur as normal matrices of least squares problems in statistics or as kernel matrices in machine learning and approximation theory. They are typically large and dense. Thus algorithms to solve systems with such a matrix can be very costly. A core idea to reduce computational complexity is to approximate the matrix by one with a low rank. The optimal and well understood choice is based on the eigenvalue decomposition of the matrix. Unfortunately, this is computationally very expensive. Cheaper methods are based on Gaussian elimination but they require pivoting. We show how invariant matrix theory provides explicit error formulas for an averaged error based on volume sampling. The formula leads to ratios of elementary symmetric polynomials on the eigenvalues. We discuss several bounds for the expected norm of the approximation error and include examples where this expected error norm can be computed exactly. References A. Dax. “On extremum properties of orthogonal quotients matrices”. In: Lin. Alg. Appl. 432.5 (2010), pp. 1234–1257. doi: 10.1016/j.laa.2009.10.034. M. Dereziński and M. W. Mahoney. Determinantal Point Processes in Randomized Numerical Linear Algebra. 2020. url: https://arxiv.org/abs/2005.03185. A. Deshpande, L. Rademacher, S. Vempala, and G. Wang. “Matrix approximation and projective clustering via volume sampling”. In: Proceedings of the Seventeenth Annual ACM-SIAM Symposium on Discrete Algorithm. SODA ’06. Miami, Florida: Society for Industrial and Applied Mathematics, 2006, pp. 1117–1126. url: https://dl.acm.org/doi/abs/10.5555/1109557.1109681. S. A. Goreinov, E. E. Tyrtyshnikov, and N. L. Zamarashkin. “A theory of pseudoskeleton approximations”. In: Lin. Alg. Appl. 261.1 (1997), pp. 1–21. doi: 10.1016/S0024-3795(96)00301-1. M. W. Mahoney and P. Drineas. “CUR matrix decompositions for improved data analysis”. In: Proc. Nat. Acad. Sci. 106.3 (Jan. 20, 2009), pp. 697–702. doi: 10.1073/pnas.0803205106. M. Marcus and L. Lopes. “Inequalities for symmetric functions and Hermitian matrices”. In: Can. J. Math. 9 (1957), pp. 305–312. doi: 10.4153/CJM-1957-037-9.

Fast and Accurate Log-Determinant Approximations

Fast and accurate log-determinant approximations

Fast Approximate Determinants Using Rational Functions

Large-scale Log-determinant Computation through Stochastic Chebyshev Expansions

Suboptimal subspace construction for log-determinant approximation

Sparse inversion for derivative of log determinant

Nearly Optimal Approximation of Matrix Functions by the Lanczos Method

Low rank approximation of positive semi-definite symmetric matrices using Gaussian elimination and volume sampling

Approximate Gaussian Elimination for Laplacians: Fast, Sparse, and Simple

Learning-Based Low-Rank Approximations

Fast $(1+\varepsilon)$-Approximation Algorithms for Binary Matrix Factorization

Adaptive Factored Incomplete Inverse Matrices

Learning Log-Determinant Divergences for Positive Definite Matrices

Good and Fast Row-Sparse ah-Symmetric Reflexive Generalized Inverses

Solving Log-Determinant Optimization Problems by a Newton-CG Primal Proximal Point Algorithm

Scalable Approximate Optimal Diagonal Preconditioning

A Proximal Point Algorithm For Log-Determinant Optimization With Group Lasso Regularization

Fast and Low Memory Cost Matrix Factorization: Algorithm, Analysis, and Case Study.

"Compress and eliminate" solver for symmetric positive definite sparse matrices

Near Optimal Frequent Directions for Sketching Dense and Sparse Matrices

An $N \log N$ Parallel Fast Direct Solver for Kernel Matrices