Abstract:High-dimensional and incomplete (HDI) data are commonly encountered in various big data-related applications concerning the complex interactions among numerous nodes, such as the user-item iterations in a recommender system. A stochastic gradient descent (SGD)-based latent factor analysis (LFA) model can perform efficient representation learning to such HDI data, thereby extracting useful knowledge from them. However, a standard SGD algorithm updates a latent factor based on the current stochastic gradient only, without the considerations on the past information, making a resultant model suffer from slow convergence. To address this critical issue, this paper proposes an Adaptive Non-linear PID-incorporated SGD (ANPS) algorithm with two-fold ideas: 1) rebuilding the instant learning error when computing the stochastic gradient following the principle of a nonlinear PID controller to incorporate past update information into the learning scheme efficiently, and 2) implementing gain parameter adaptation following the principle of particle swarm optimization (PSO). Experiments on six widely-adopted HDI datasets demonstrate that compared with state-of-the-art LFA models, an ANPS-based LFA model achieves significant advantage in both efficiency and accuracy. Moreover, its flexible gain parameter adaptation mechanism greatly boosts its practicability for real issues. Note to Practitioners—In many industrial applications like recommender systems, social network systems, and cloud service systems, people usually encounter numerous nodes and their highly-incomplete relationships. An HDI matrix is commonly adopted to describe such specific relationships. One of the major challenges is to acquire useful knowledge from an HDI matrix efficiently and accurately for various data analysis tasks, e.g., accurate recommendation, community detection, and web service selection. An SGD-based LFA model has been widely adopted to tackle this issue. However, it suffers from slow convergence that leads to considerable time cost on large-scale datasets. This study proposes an ANPS algorithm following the principle of a nonlinear PID controller. With it, an ANPS-based LFA model is achieved, which possesses fast convergence rate on an industrial HDI matrix. The proposed ANPS algorithm can be leveraged for different types of various machine learning models, thereby improving their utility and scalability in practice.

Hyper-parameter-evolutionary latent factor analysis for high-dimensional and sparse data from recommender systems

Hierarchical Particle Swarm Optimization-incorporated Latent Factor Analysis for Large-Scale Incomplete Matrices

Hyper-Parameter-Free Latent Factor Analysis via Adaptive Moment Estimation-Incorporated Particle Swarm Optimization

Position-Transitional Particle Swarm Optimization-incorporated Latent Factor Analysis

A proportional-integral-derivative-incorporated stochastic gradient descent-based latent factor analysis model

A Game-Incorporated Alternative Least Squares-Based Approach to Latent Factor Analysis

Adaptive Regularization-Incorporated Latent Factor Analysis

A State-Migration Particle Swarm Optimizer for Adaptive Latent Factor Analysis of High-Dimensional and Incomplete Data

Adaptive Latent Factor Analysis via Generalized Momentum-Incorporated Particle Swarm Optimization

An Adam-enhanced Particle Swarm Optimizer for Latent Factor Analysis

A Nonlinear PID-Incorporated Adaptive Stochastic Gradient Descent Algorithm for Latent Factor Analysis

An α–β-Divergence-Generalized Recommender for Highly Accurate Predictions of Missing User Preferences

A Dynamic-Neighbor Particle Swarm Optimizer for Accurate Latent Factor Analysis

Pseudo Gradient-Adjusted Particle Swarm Optimization for Accurate Adaptive Latent Factor Analysis

An L1-and-L2-Norm-Oriented Latent Factor Model for Recommender Systems

PMLF: Prediction-Sampling-based Multilayer-Structured Latent Factor Analysis

A Multilayered-and-Randomized Latent Factor Model for High-Dimensional and Sparse Matrices

Algorithms of Unconstrained Non-Negative Latent Factor Analysis for Recommender Systems.

A Generalized and Fast-converging Non-negative Latent Factor Model for Predicting User Preferences in Recommender Systems.

Large-scale and Scalable Latent Factor Analysis via Distributed Alternative Stochastic Gradient Descent for Recommender Systems

A Nonlinear PID-Enhanced Adaptive Latent Factor Analysis Model