Abstract:Collaborative Filtering (CF) methods dominate real-world recommender systems given their ability to learn high-quality, sparse ID-embedding tables that effectively capture user preferences. These tables scale linearly with the number of users and items, and are trained to ensure high similarity between embeddings of interacted user-item pairs, while maintaining low similarity for non-interacted pairs. Despite their high performance, encouraging dispersion for non-interacted pairs necessitates expensive regularization (e.g., negative sampling), hurting runtime and scalability. Existing research tends to address these challenges by simplifying the learning process, either by reducing model complexity or sampling data, trading performance for runtime. In this work, we move beyond model-level modifications and study the properties of the embedding tables under different learning strategies. Through theoretical analysis, we find that the singular values of the embedding tables are intrinsically linked to different CF loss functions. These findings are empirically validated on real-world datasets, demonstrating the practical benefits of higher stable rank, a continuous version of matrix rank which encodes the distribution of singular values. Based on these insights, we propose an efficient warm-start strategy that regularizes the stable rank of the user and item embeddings. We show that stable rank regularization during early training phases can promote higher-quality embeddings, resulting in training speed improvements of up to 66%. Additionally, stable rank regularization can act as a proxy for negative sampling, allowing for performance gains of up to 21% over loss functions with small negative sampling ratios. Overall, our analysis unifies current CF methods under a new perspective, their optimization of stable rank, motivating a flexible regularization method.

Preference Completion: Large-scale Collaborative Ranking from Pairwise Comparisons

Multiple Pairwise Ranking with Implicit Feedback.

Collaborative Rating Allocation.

ILCR: Item-based Latent Factors for Sparse Collaborative Retrieval

Online Low Rank Matrix Completion

Fair Active Ranking from Pairwise Preferences

Ranking with Confidence for Large Scale Comparison Data

SetRank: A Setwise Bayesian Approach for Collaborative Ranking from Implicit Feedback

Inductive Pairwise Ranking: Going Beyond the n log(n) Barrier

A Harmonic Extension Approach for Collaborative Ranking

Online Matrix Completion: A Collaborative Approach with Hott Items

Collaborative Deep Ranking: A Hybrid Pair-Wise Recommendation Algorithm with Implicit Feedback

Understanding and Scaling Collaborative Filtering Optimization from the Perspective of Matrix Rank

Learning-Efficient Yet Generalizable Collaborative Filtering for Item Recommendation

Ranking-Oriented Collaborative Filtering

Double bayesian pairwise learning for one-class collaborative filtering

Stochastic Iterative Methods for Online Rank Aggregation from Pairwise Comparisons

Adaptive Pairwise Preference Learning for Collaborative Recommendation with Implicit Feedbacks.

Approximate Ranking from Pairwise Comparisons

Collaborative Ranking with Ranking-Based Neighborhood

Adaptive Matrix Completion for the Users and the Items in Tail