Abstract:Matrix factorization (MF) is a widely used collaborative filtering (CF) algorithm for recommendation systems (RSs), due to its high prediction accuracy, great flexibility and high efficiency in big data processing. However, with the dramatically increased number of users/items in current RSs, the computational complexity for training a MF model largely increases. Many existing works have accelerated MF, by either putting in additional computational resources or utilizing parallel systems, introducing a large cost. In this paper, we propose algorithmic methods to accelerate MF, without inducing any additional computational resources. In specific, we observe fine-grained structured sparsity in the decomposed feature matrices when considering a certain threshold. The fine-grained structured sparsity causes a large amount of unnecessary operations during both matrix multiplication and latent factor update, increasing the computational time of the MF training process. Based on the observation, we firstly propose to rearrange the feature matrices based on joint sparsity, which potentially makes a latent vector with a smaller index more dense than that with a larger index. The feature matrix rearrangement is given to limit the error caused by the later performed pruning process. We then propose to prune the insignificant latent factors by an early stopping process during both matrix multiplication and latent factor update. The pruning process is dynamically performed according to the sparsity of the latent factors for different users/items, to accelerate the process. The experiments show that our method can achieve 1.2-1.65 speedups, with up to 20.08% error increase, compared with the conventional MF training process. We also prove the proposed methods are applicable considering different hyperparameters including optimizer, optimization strategy and initialization method.

Matrix Polynomial Predictive Model: A New Approach to Accelerating the PARAFAC Decomposition.

Convergence Acceleration of Alternating Least Squares with a Matrix Polynomial Predictive Model for PARAFAC Decomposition of a Tensor

An Alternating Bayesian Approach to PARAFAC Decomposition of Tensors.

PSL - Exploiting Parallelism, Sparsity and Locality to Accelerate Matrix Factorization on x86 Platforms.

Towards More Efficient SPSD Matrix Approximation and CUR Matrix Decomposition

Mixed-Precision Paterson--Stockmeyer Method for Evaluating Polynomials of Matrices

Efficient Acceleration Framework for Complex‐Valued Linear Systems: Alternating Anderson Accelerating Preconditioned Richardson Approach

Accelerating Matrix Factorization by Dynamic Pruning for Fast Recommendation

PARAFAC2-based Coupled Matrix and Tensor Factorizations with Constraints

Acceleration of Approximate Matrix Multiplications on GPUs

MPMA: Mixture Probabilistic Matrix Approximation for Collaborative Filtering

Accelerated PALM for Nonconvex Low-Rank Matrix Recovery with Theoretical Analysis

Parallel model order reduction based on block discrete Fourier transform and Krylov subspace for parametric systems

Accelerating regularized tensor decomposition using the alternating direction method of multipliers with multiple Nesterov's extrapolations

Accurate Low-Degree Polynomial Approximation of Non-polynomial Operators for Fast Private Inference in Homomorphic Encryption

A fast Multiplicative Updates algorithm for Non-negative Matrix Factorization

Fast and backward stable computation of the eigenvalues and eigenvectors of matrix polynomials

Fast and Low Memory Cost Matrix Factorization: Algorithm, Analysis, and Case Study.

Algorithmic Acceleration of Parallel ALS for Collaborative Filtering: Speeding up Distributed Big Data Recommendation in Spark

A NEW MODEL FOR SPARSE AND LOW-RANK MATRIX DECOMPOSITION

BALS: Blocked Alternating Least Squares for Parallel Sparse Matrix Factorization on GPUs