Abstract:Collaborative Filtering (CF) methods dominate real-world recommender systems given their ability to learn high-quality, sparse ID-embedding tables that effectively capture user preferences. These tables scale linearly with the number of users and items, and are trained to ensure high similarity between embeddings of interacted user-item pairs, while maintaining low similarity for non-interacted pairs. Despite their high performance, encouraging dispersion for non-interacted pairs necessitates expensive regularization (e.g., negative sampling), hurting runtime and scalability. Existing research tends to address these challenges by simplifying the learning process, either by reducing model complexity or sampling data, trading performance for runtime. In this work, we move beyond model-level modifications and study the properties of the embedding tables under different learning strategies. Through theoretical analysis, we find that the singular values of the embedding tables are intrinsically linked to different CF loss functions. These findings are empirically validated on real-world datasets, demonstrating the practical benefits of higher stable rank, a continuous version of matrix rank which encodes the distribution of singular values. Based on these insights, we propose an efficient warm-start strategy that regularizes the stable rank of the user and item embeddings. We show that stable rank regularization during early training phases can promote higher-quality embeddings, resulting in training speed improvements of up to 66%. Additionally, stable rank regularization can act as a proxy for negative sampling, allowing for performance gains of up to 21% over loss functions with small negative sampling ratios. Overall, our analysis unifies current CF methods under a new perspective, their optimization of stable rank, motivating a flexible regularization method.

ImplicitSLIM and How it Improves Embedding-based Collaborative Filtering

Using latent class models for neighbors selection in collaborative filtering

Confidence-Learning Based Collaborative Filtering With Heterogeneous Implicit Feedbacks

Improving Recommendations With Collaborative Factors

A unified latent factor correction scheme for collaborative filtering

Deep Item-based Collaborative Filtering for Sparse Implicit Feedback

Online Collaborative Filtering With Implicit Feedback

Sparse Bayesian Content-Aware Collaborative Filtering for Implicit Feedback

ILCR: Item-based Latent Factors for Sparse Collaborative Retrieval

Deep Collaborative Filtering Incorporating Auxiliary Multi-Media Information.

Understanding and Scaling Collaborative Filtering Optimization from the Perspective of Matrix Rank

Incremental Learning for Dynamic Collaborative Filtering.

Fusion Recommendation Algorithm Based on DeepWalk and Item-Based Collaborative Filtering

Smoothing based approach for hybrid collaborative filtering

Fast Collaborative Filtering from Implicit Feedback with Provable Guarantees

Sparse Linear Capsules for Matrix Factorization-Based Collaborative Filtering.

Collaborative filtering with sequential implicit feedback via learning users' preferences over item-sets

Learning to Recommend from Sparse Data via Generative User Feedback

Latent Factor Interpretations for Collaborative Filtering

Improving One-Class Collaborative Filtering Via Ranking-Based Implicit Regularizer.

Consistent Collaborative Filtering via Tensor Decomposition