Abstract:Contrastive Learning (CL) has achieved impressive performance in self-supervised learning tasks, showing superior generalization ability. Inspired by the success, adopting CL into collaborative filtering (CF) is prevailing in semi-supervised top-K recommendations. The basic idea is to routinely conduct heuristic-based data augmentation and apply contrastive losses (e.g., InfoNCE) on the augmented views. Yet, some CF-tailored challenges make this adoption suboptimal, such as the issue of out-of-distribution, the risk of false negatives, and the nature of top-K evaluation. They necessitate the CL-based CF scheme to focus more on mining hard negatives and distinguishing false negatives from the vast unlabeled user-item interactions, for informative contrast signals. Worse still, there is limited understanding of contrastive loss in CF methods, especially w.r.t. its generalization ability. To bridge the gap, we delve into the reasons underpinning the success of contrastive loss in CF, and propose a principled Adversarial InfoNCE loss (AdvInfoNCE), which is a variant of InfoNCE, specially tailored for CF methods. AdvInfoNCE adaptively explores and assigns hardness to each negative instance in an adversarial fashion and further utilizes a fine-grained hardness-aware ranking criterion to empower the recommender's generalization ability. Training CF models with AdvInfoNCE, we validate the effectiveness of AdvInfoNCE on both synthetic and real-world benchmark datasets, thus showing its generalization ability to mitigate out-of-distribution problems. Given the theoretical guarantees and empirical superiority of AdvInfoNCE over most contrastive loss functions, we advocate its adoption as a standard loss in recommender systems, particularly for the out-of-distribution tasks. Codes are available at <a class="link-external link-https" href="https://github.com/LehengTHU/AdvInfoNCE" rel="external noopener nofollow">this https URL</a>.

Towards noise contrastive estimation with soft targets for conditional models

SINCERE: Supervised Information Noise-Contrastive Estimation REvisited

Taming the Cross Entropy Loss

Can Cross Entropy Loss Be Robust to Label Noise?

Mitigating Memorization of Noisy Labels by Clipping the Model Prediction

Latent Class-Conditional Noise Model

Robust Contrastive Learning against Noisy Views

Contrastive Learning Improves Model Robustness Under Label Noise

InfoNCE: Identifying the Gap Between Theory and Practice

Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels

On the connection between Noise-Contrastive Estimation and Contrastive Divergence

Fully Variational Noise-Contrastive Estimation

Asymmetric Loss Functions for Noise-Tolerant Learning: Theory and Applications

Rethinking InfoNCE: How Many Negative Samples Do You Need?

Learning with Noisy Labels Via Sparse Regularization

Noise-Robust Loss Functions: Enhancing Bounded Losses for Large-Scale Noisy Data Learning

Ranking Info Noise Contrastive Estimation: Boosting Contrastive Learning via Ranked Positives

Supervised Contrastive Learning with Hard Negative Samples

Towards a Unified Framework of Contrastive Learning for Disentangled Representations

Soft (Gaussian CDE) regression models and loss functions

Empowering Collaborative Filtering with Principled Adversarial Contrastive Loss