Abstract:The element-wise functions of a matrix are widely used in machine learning. For the applications with large matrices, efficiently computing the matrix-vector multiplication of matrix element-wise function without explicitly constructed matrix is very desired. In this work, we aim to develop an efficient low-rank approximation of the element-wise function of matrix with the time/memory cost linear to the matrix dimension. We first propose a sparse-sign streaming randomized SVD (ssrSVD) algorithm based on a streaming singular value decomposition (SVD) algorithm and the sparse-sign random projection for the approximation of element-wise function of general asymmetric matrix. For symmetric positive semi-definite (SPSD) matrix, for which the existing Nyström [1] and FastSPSD [2] method do not perform well if the matrix's singular value decays slowly, we propose a theoretically proved shift skill to improve the approximation accuracy. Combining with the ssrSVD, we obtain the sparse-sign streaming SPSD matrix approximation with shift (S3SPSD) algorithm. Experiments are carried out to evaluate the proposed algorithms' performance in approximating element-wise functions of matrix. With the color transfer task based on the Sinkhorn algorithm, the ssrSVD algorithm largely reduces the approximation error (up to $10^{5}\times$) compared with the state-of-the-art baselines, and results in high-quality color transfer result. For the kernel matrix approximation, the proposed S3SPSD algorithm also consistently outperforms the state-of-the-art baselines. Experimental results finally validate the linear time complexity of the proposed algorithms.

A Fast Algorithm for Unitary ESPRIT

Unitary Approximate Message Passing for Sparse Bayesian Learning

Efficient Channel Estimation for RIS-Aided MIMO Communications with Unitary Approximate Message Passing

A Novel Unitary ESPRIT Algorithm for Monostatic FDA-MIMO Radar

Two types of fast ESPRIT algorithms

A Fast deflation Method for Sparse Principal Component Analysis via Subspace Projections

Large-Dimensional Positive Definite Covariance Estimation for High Frequency Data via Low-rank and Sparse Matrix Decomposition

On the exact spectral factorization of rational matrix functions with applications to paraunitary filter banks

Sparse Spatial Spectral Fitting with Nonuniform Noise Covariance Matrix Estimation Based on Semidefinite Optimization

A HOOI-Based Fast Parameter Estimation Algorithm in UCA-UCFO Framework

A Novel Single Snapshot Two-Dimensional ESPRIT Algorithm

ESPRIT-estimation of signal parameters via rotational invariance techniques

Low-complexity Method for DOA Estimation Based on ESPRIT

Unitary Approximate Message Passing for Matrix Factorization

Approximating Element-Wise Functions of Matrix with Improved Streaming Randomized SVD

Exploiting Structured Sparsity with Low Complexity Sparse Bayesian Learning for RIS-assisted MIMO Channel Estimation

Singular Value Decomposition For Unitary Symmetric Matrix

Fast implementation of sparse iterative covariance-based estimation for processing MST radar data

High-resolution estimation of frequencies, azimuths and elevations of coherent signals

SSP-Based UBI Algorithms for Uniform Linear Array

Accelerating Iteratively Linear Detectors in Multi-User (ELAA-)MIMO Systems with UW-SVD