Abstract:Contrastive Learning (CL) has achieved impressive performance in self-supervised learning tasks, showing superior generalization ability. Inspired by the success, adopting CL into collaborative filtering (CF) is prevailing in semi-supervised top-K recommendations. The basic idea is to routinely conduct heuristic-based data augmentation and apply contrastive losses (e.g., InfoNCE) on the augmented views. Yet, some CF-tailored challenges make this adoption suboptimal, such as the issue of out-of-distribution, the risk of false negatives, and the nature of top-K evaluation. They necessitate the CL-based CF scheme to focus more on mining hard negatives and distinguishing false negatives from the vast unlabeled user-item interactions, for informative contrast signals. Worse still, there is limited understanding of contrastive loss in CF methods, especially w.r.t. its generalization ability. To bridge the gap, we delve into the reasons underpinning the success of contrastive loss in CF, and propose a principled Adversarial InfoNCE loss (AdvInfoNCE), which is a variant of InfoNCE, specially tailored for CF methods. AdvInfoNCE adaptively explores and assigns hardness to each negative instance in an adversarial fashion and further utilizes a fine-grained hardness-aware ranking criterion to empower the recommender's generalization ability. Training CF models with AdvInfoNCE, we validate the effectiveness of AdvInfoNCE on both synthetic and real-world benchmark datasets, thus showing its generalization ability to mitigate out-of-distribution problems. Given the theoretical guarantees and empirical superiority of AdvInfoNCE over most contrastive loss functions, we advocate its adoption as a standard loss in recommender systems, particularly for the out-of-distribution tasks. Codes are available at <a class="link-external link-https" href="https://github.com/LehengTHU/AdvInfoNCE" rel="external noopener nofollow">this https URL</a>.

SINCERE: Supervised Information Noise-Contrastive Estimation REvisited

Ranking Info Noise Contrastive Estimation: Boosting Contrastive Learning via Ranked Positives

Perfectly Balanced: Improving Transfer and Robustness of Supervised Contrastive Learning

Improving Contrastive Learning of Sentence Embeddings with Focal-InfoNCE

A Unified Contrastive Loss for Self-Training

InfoNCE: Identifying the Gap Between Theory and Practice

Revisiting Recommendation Loss Functions through Contrastive Learning (Technical Report)

Towards noise contrastive estimation with soft targets for conditional models

Rethinking InfoNCE: How Many Negative Samples Do You Need?

Adversarial Contrastive Learning via Asymmetric InfoNCE.

Contrastive Feature Loss for Image Prediction

Empowering Collaborative Filtering with Principled Adversarial Contrastive Loss

Generalized Supervised Contrastive Learning

SimCSE++: Improving Contrastive Learning for Sentence Embeddings from Two Perspectives

Towards a rigorous analysis of mutual information in contrastive learning

Engineering the Neural Collapse Geometry of Supervised-Contrastive Loss

Leveraging Superfluous Information in Contrastive Representation Learning

Semantic-Enhanced Supervised Contrastive Learning.

SCNet: semi-supervised and contrastive learning against noisy labels with two selection strategy approach

Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss

Noisy Correspondence Learning with Self-Reinforcing Errors Mitigation