Abstract:Partial eigenvalue decomposition (PEVD) and partial singular value decomposition (PSVD) of large sparse matrices are of fundamental importance in a wide range of applications, including latent semantic indexing, spectral clustering, and kernel methods for machine learning. The more challenging problems are when a large number of eigenpairs or singular triplets need to be computed. We develop practical and efficient algorithms for these challenging problems. Our algorithms are based on a filter-accelerated block Davidson method. Two types of filters are utilized, one is Chebyshev polynomial filtering, the other is rational-function filtering by solving linear equations. The former utilizes the fastest growth of the Chebyshev polynomial among same degree polynomials; the latter employs the traditional idea of shift-invert, for which we address the important issue of automatic choice of shifts and propose a practical method for solving the shifted linear equations inside the block Davidson method. Our two filters can efficiently generate high-quality basis vectors to augment the projection subspace at each Davidson iteration step, which allows a restart scheme using an active projection subspace of small dimension. This makes our algorithms memory-economical, thus practical for large PEVD/PSVD calculations. We compare our algorithms with representative methods, including ARPACK, PROPACK, the randomized SVD method, and the limited memory SVD method. Extensive numerical tests on representative datasets demonstrate that, in general, our methods have similar or faster convergence speed in terms of CPU time, while requiring much lower memory comparing with other methods. The much lower memory requirement makes our methods more practical for large-scale PEVD/PSVD computations.

Efficient Asynchronous Semi-Stochastic Block Coordinate Descent Methods for Large-Scale SVD

Efficient Parallel Stochastic Variance Reduction Algorithms for Large-Scale SVD

Asynchronous Stochastic Block Coordinate Descent with Variance Reduction

Accelerated Stochastic ADMM with Variance Reduction

Limited Memory Block Krylov Subspace Optimization for Computing Dominant Singular Value Decompositions.

Solving Large-Scale Support Vector Ordinal Regression with Asynchronous Parallel Coordinate Descent Algorithms.

A Novel Fully Hardware-Implemented SVD Solver Based on Ultra-Parallel BCV Jacobi Algorithm

On the Flexibility of Block Coordinate Descent for Large-Scale Optimization.

Stochastic Parallel Block Coordinate Descent for Large-scale Saddle Point Problems

Accelerated Variance Reduced Block Coordinate Descent

Coordinate Descent Algorithms for Large-Scale SVDD

Large-scale Nyström kernel matrix approximation using randomized SVD.

Bilateral Sampling Randomized Singular Value Decomposition

Scalable Dual Coordinate Descent for Kernel Methods

Accelerating large partial EVD/SVD calculations by filtered block Davidson methods

Approximating Element-Wise Functions of Matrix with Improved Streaming Randomized SVD

The Application of ACA and SVD Algorithms in the Matrix Low-Rank Approximation

Asynchronous Parallel, Sparse Approximated SVRG for High-Dimensional Machine Learning

Accelerated Variance Reduction Stochastic ADMM for Large-Scale Machine Learning

W-Cycle SVD: A Multilevel Algorithm for Batched SVD on GPUs

Asynchronous Doubly Stochastic Proximal Optimization with Variance Reduction.