Abstract:Contrastive Learning (CL) has emerged as a dominant technique for unsupervised representation learning which embeds augmented versions of the anchor close to each other (positive samples) and pushes the embeddings of other samples (negatives) apart. As revealed in recent studies, CL can benefit from hard negatives (negatives that are most similar to the anchor). However, we observe limited benefits when we adopt existing hard negative mining techniques of other domains in Graph Contrastive Learning (GCL). We perform both experimental and theoretical analysis on this phenomenon and find it can be attributed to the message passing of Graph Neural Networks (GNNs). Unlike CL in other domains, most hard negatives are potentially false negatives (negatives that share the same class with the anchor) if they are selected merely according to the similarities between anchor and themselves, which will undesirably push away the samples of the same class. To remedy this deficiency, we propose an effective method, dubbed \textbf{ProGCL}, to estimate the probability of a negative being true one, which constitutes a more suitable measure for negatives' hardness together with similarity. Additionally, we devise two schemes (i.e., \textbf{ProGCL-weight} and \textbf{ProGCL-mix}) to boost the performance of GCL. Extensive experiments demonstrate that ProGCL brings notable and consistent improvements over base GCL methods and yields multiple state-of-the-art results on several unsupervised benchmarks or even exceeds the performance of supervised ones. Also, ProGCL is readily pluggable into various negatives-based GCL methods for performance improvement. We release the code at \textcolor{magenta}{\url{https://github.com/junxia97/ProGCL}}.

M-Mix: Generating Hard Negatives via Multi-sample Mixing for Contrastive Learning

DropMix: Better Graph Contrastive Learning with Harder Negative Samples

ProGCL: Rethinking Hard Negative Mining in Graph Contrastive Learning

Debiased Graph Contrastive Learning.

MixGCF

MixKG: Mixing for harder negative samples in knowledge graph

ANOMIX: A Simple yet Effective Hard Negative Generation via Mixing for Graph Anomaly Detection

Unsupervised Sentence Representation Via Contrastive Learning with Mixing Negatives

Synthetic Hard Negative Samples for Contrastive Learning

Un-mix: Rethinking Image Mixtures for Unsupervised Visual Representation Learning

M2ixKG: Mixing for harder negative samples in knowledge graph

MixCL: Mixed Contrastive Learning for Relation Extraction

Negative Samples Mining Matters: Reconsidering Hyperspectral Image Classification with Contrastive Learning

Multiplex Graph Contrastive Learning with Soft Negatives

Harnessing Hard Mixed Samples with Decoupled Regularizer

SMMix: Self-Motivated Image Mixing for Vision Transformers

SyNeg: LLM-Driven Synthetic Hard-Negatives for Dense Retrieval

Progressive Hard Negative Masking: from Global Uniformity to Local Tolerance

Generating Counterfactual Hard Negative Samples for Graph Contrastive Learning

An Adaptive Mixup Hard Negative Sampling for Zero-Shot Entity Linking

SUMix: Mixup with Semantic and Uncertain Information