Abstract:Matrix factorization (MF) is a widely used collaborative filtering (CF) algorithm for recommendation systems (RSs), due to its high prediction accuracy, great flexibility and high efficiency in big data processing. However, with the dramatically increased number of users/items in current RSs, the computational complexity for training a MF model largely increases. Many existing works have accelerated MF, by either putting in additional computational resources or utilizing parallel systems, introducing a large cost. In this paper, we propose algorithmic methods to accelerate MF, without inducing any additional computational resources. In specific, we observe fine-grained structured sparsity in the decomposed feature matrices when considering a certain threshold. The fine-grained structured sparsity causes a large amount of unnecessary operations during both matrix multiplication and latent factor update, increasing the computational time of the MF training process. Based on the observation, we firstly propose to rearrange the feature matrices based on joint sparsity, which potentially makes a latent vector with a smaller index more dense than that with a larger index. The feature matrix rearrangement is given to limit the error caused by the later performed pruning process. We then propose to prune the insignificant latent factors by an early stopping process during both matrix multiplication and latent factor update. The pruning process is dynamically performed according to the sparsity of the latent factors for different users/items, to accelerate the process. The experiments show that our method can achieve 1.2-1.65 speedups, with up to 20.08% error increase, compared with the conventional MF training process. We also prove the proposed methods are applicable considering different hyperparameters including optimizer, optimization strategy and initialization method.

PSL - Exploiting Parallelism, Sparsity and Locality to Accelerate Matrix Factorization on x86 Platforms.

Privacy Preserving Point-of-Interest Recommendation Using Decentralized Matrix Factorization.

BALS: Blocked Alternating Least Squares for Parallel Sparse Matrix Factorization on GPUs

A Fast Parallel Stochastic Gradient Method for Matrix Factorization in Shared Memory Systems

Performance Optimization for Sparse A(T)Ax in Parallel on Multicore Cpu

Algorithmic Acceleration of Parallel ALS for Collaborative Filtering: Speeding up Distributed Big Data Recommendation in Spark

Discrete Social Recommendation.

Fast and Low Memory Cost Matrix Factorization: Algorithm, Analysis, and Case Study.

Accelerating Matrix Factorization by Dynamic Pruning for Fast Recommendation

Accelerated Structured Matrix Factorization

A Highly Efficient GPU-CPU Hybrid Parallel Implementation of Sparse LU Factorization

On the Parallel I/O Optimality of Linear Algebra Kernels: Near-Optimal Matrix Factorizations

Skew-Symmetric Matrix Decompositions on Shared-Memory Architectures

Matrix Polynomial Predictive Model: A New Approach to Accelerating the PARAFAC Decomposition.

ALX: Large Scale Matrix Factorization on TPUs

Localized matrix factorization for recommendation based on matrix block diagonal forms

Computing Low-Rank Approximation of a Dense Matrix on Multicore CPUs with a GPU and Its Application to Solving a Hierarchically Semiseparable Linear System of Equations

$O(N)$ distributed direct factorization of structured dense matrices using runtime systems

Parallel Cholesky Factorization for Banded Matrices using OpenMP Tasks

RankMat : Matrix Factorization with Calibrated Distributed Embedding and Fairness Enhancement

Accelerating Sparse Cholesky Factorization on Sunway Manycore Architecture.