Abstract:This paper aims to investigate efficient and scalable machine learning algorithms for resolving Non-negative Matrix Factorization (NMF), which is important for many real-world applications, particularly for collaborative filtering and recommender systems. Unlike traditional batch learning methods, a recently proposed online learning technique named "NN-PA" tackles NMF by applying the popular Passive-Aggressive (PA) online learning, and found promising results. Despite its simplicity and high efficiency, NN-PA falls short in at least two critical limitations: (i) it only exploits the first-order information and thus may converge slowly especially at the beginning of online learning tasks; (ii) it is sensitive to some key parameters which are often difficult to be tuned manually, particularly in a practical online learning system. In this work, we present a novel family of online Adaptive Passive-Aggressive (APA) learning algorithms for NMF, named "NN-APA", which overcomes two critical limitations of NN-PA by (i) exploiting second-order information to enhance PA in making more informative updates at each iteration; and (ii) achieving the parameter auto-selection by exploring the idea of online learning with expert advice in deciding the optimal combination of the key parameters in NMF. We theoretically analyze the regret bounds of the proposed method and show its advantage over the state-of-the-art NN-PA method, and further validate the efficacy and scalability of the proposed technique through an extensive set of experiments on a variety of large-scale real recommender systems datasets.

On adaptively accelerated Arnoldi method for computing PageRank

Accelerating the Arnoldi-Type Algorithm for the PageRank Problem and the ProteinRank Problem

An Arnoldi-Extrapolation algorithm for computing PageRank

A Power–Arnoldi Algorithm for Computing PageRank

Arnoldi Versus GMRES for Computing Pagerank

Enhanced preprocessed multi-step splitting iterations for computing PageRank

Weak dangling block reordering and multi-step block compression for efficiently computing and updating PageRank solutions

Shifted power-GMRES method accelerated by extrapolation for solving PageRank with multiple damping factors

A VARIANT OF THE REFINED ARNOLDI METHOD FOR LARGE SCALE MATRIX EIGENPROBLEMS

Efficient Algorithms for Personalized PageRank

Accelerating Personalized PageRank Vector Computation

Distributed PageRank Computation Based on Iterative Aggregation-Disaggregation Methods.

Randomized Incremental Algorithms For The Pagerank Computation

A Survey on Personalized PageRank Computation Algorithms

Online Adaptive Passive-Aggressive Methods for Non-Negative Matrix Factorization and Its Applications

Two Accelerated Non-backtracking PageRank Algorithms for Large-scale Networks

Weighted P-Rank Algorithm Based on aHeterogeneous Scholarly Network

Efficient GPU Implementation of Static and Incrementally Expanding DF-P PageRank for Dynamic Graphs

Fast Computation of Stationary Joint Probability Distribution of Sparse Markov Chains

Efficient Algorithms for Personalized PageRank Computation: A Survey

Efficient PageRank and SpMV Computation on AMD GPUs