Abstract:Collaborative Filtering (CF) methods dominate real-world recommender systems given their ability to learn high-quality, sparse ID-embedding tables that effectively capture user preferences. These tables scale linearly with the number of users and items, and are trained to ensure high similarity between embeddings of interacted user-item pairs, while maintaining low similarity for non-interacted pairs. Despite their high performance, encouraging dispersion for non-interacted pairs necessitates expensive regularization (e.g., negative sampling), hurting runtime and scalability. Existing research tends to address these challenges by simplifying the learning process, either by reducing model complexity or sampling data, trading performance for runtime. In this work, we move beyond model-level modifications and study the properties of the embedding tables under different learning strategies. Through theoretical analysis, we find that the singular values of the embedding tables are intrinsically linked to different CF loss functions. These findings are empirically validated on real-world datasets, demonstrating the practical benefits of higher stable rank, a continuous version of matrix rank which encodes the distribution of singular values. Based on these insights, we propose an efficient warm-start strategy that regularizes the stable rank of the user and item embeddings. We show that stable rank regularization during early training phases can promote higher-quality embeddings, resulting in training speed improvements of up to 66%. Additionally, stable rank regularization can act as a proxy for negative sampling, allowing for performance gains of up to 21% over loss functions with small negative sampling ratios. Overall, our analysis unifies current CF methods under a new perspective, their optimization of stable rank, motivating a flexible regularization method.

Sparse Online Collaborative Filtering with Dynamic Regularization

Using latent class models for neighbors selection in collaborative filtering

Improving Recommendations With Collaborative Factors

Enhanced Knowledge Transfer for Collaborative Filtering with Multi-Source Heterogeneous Feedbacks

Online Collaborative Filtering With Implicit Feedback

Recommendation with Item Clustering Based Collaborative Filtering

Sparse Online Learning for Collaborative Filtering.

Improving One-Class Collaborative Filtering Via Ranking-Based Implicit Regularizer.

Information Filtering Via Collaborative User Clustering Modeling.

Deep Probabilistic Matrix Factorization Framework for Online Collaborative Filtering.

Online Multi-Task Collaborative Filtering for On-the-fly Recommender Systems

Understanding and Scaling Collaborative Filtering Optimization from the Perspective of Matrix Rank

An Efficient Collaborative Filtering Approach Using Smoothing and Fusing

Incremental Learning for Dynamic Collaborative Filtering.

Exploit Latent Dirichlet Allocation for Collaborative Filtering

Online Recommender System Based on Social Network Regularization.

Collaborative Recommendation Based on Social Community Detection

A new collaborative filtering approach utilizing item’s popularity

Kernel-CF: Collaborative filtering done right with social network analysis and kernel smoothing

Dynamic Reordering Within the Nearest Neighbor-based Algorithm for Collaborative Filtering

Scalable collaborative filtering using cluster-based smoothing.