Abstract:The element-wise functions of a matrix are widely used in machine learning. For the applications with large matrices, efficiently computing the matrix-vector multiplication of matrix element-wise function without explicitly constructed matrix is very desired. In this work, we aim to develop an efficient low-rank approximation of the element-wise function of matrix with the time/memory cost linear to the matrix dimension. We first propose a sparse-sign streaming randomized SVD (ssrSVD) algorithm based on a streaming singular value decomposition (SVD) algorithm and the sparse-sign random projection for the approximation of element-wise function of general asymmetric matrix. For symmetric positive semi-definite (SPSD) matrix, for which the existing Nyström [1] and FastSPSD [2] method do not perform well if the matrix's singular value decays slowly, we propose a theoretically proved shift skill to improve the approximation accuracy. Combining with the ssrSVD, we obtain the sparse-sign streaming SPSD matrix approximation with shift (S3SPSD) algorithm. Experiments are carried out to evaluate the proposed algorithms' performance in approximating element-wise functions of matrix. With the color transfer task based on the Sinkhorn algorithm, the ssrSVD algorithm largely reduces the approximation error (up to $10^{5}\times$) compared with the state-of-the-art baselines, and results in high-quality color transfer result. For the kernel matrix approximation, the proposed S3SPSD algorithm also consistently outperforms the state-of-the-art baselines. Experimental results finally validate the linear time complexity of the proposed algorithms.

Faster Matrix Completion Using Randomized SVD

Accelerated singular value thresholding for matrix completion.

A Fast Implementation of Singular Value Thresholding Algorithm using Recycling Rank Revealing Randomized Singular Value Decomposition

Matrix Completion and Low-Rank SVD via Fast Alternating Least Squares

A Fast Spectral Relaxation Approach to Matrix Completion Via Kronecker Products.

Approximating Element-Wise Functions of Matrix with Improved Streaming Randomized SVD

Fast Updating Truncated SVD for Representation Learning with Sparse Matrices

Fast Methods for Denoising Matrix Completion Formulations, with Applications to Robust Seismic Data Interpolation

Pass-Efficient Randomized SVD with Boosted Accuracy

Algorithm xxx: Faster Randomized SVD with Dynamic Shifts

Accelerated and Inexact Soft-Impute for Large-Scale Matrix and Tensor Completion

Fast and Accurate Matrix Completion Via Truncated Nuclear Norm Regularization

A Missing Data Reconstruction Method Using an Accelerated Least-Squares Approximation with Randomized SVD

Row-aware Randomized SVD with applications

Efficient GPU implementation of randomized SVD and its applications

Limited Memory Block Krylov Subspace Optimization for Computing Dominant Singular Value Decompositions.

Low rank matrix completion by alternating steepest descent methods

Fast randomized algorithms for computing the generalized tensor SVD based on the tubal product

A Close Look at Matrix Completion with Noisy Data

Matrix Completion Based on Low-Rank and Local Features Applied to Images Recovery and Recommendation Systems.