Abstract:Graph Contrastive Learning (GCL) excels at managing noise and fluctuations in input data, making it popular in various fields (e.g., social networks, and knowledge graphs). Our study finds that the difference in high-frequency information between augmented graphs is greater than that in low-frequency information. However, most existing GCL methods focus mainly on the time domain (low-frequency information) for node feature representations and cannot make good use of high-frequency information to speed up model convergence. Furthermore, existing GCL paradigms optimize graph embedding representations by pulling the distance between positive sample pairs closer and pushing the distance between positive and negative sample pairs farther away, but our theoretical analysis shows that graph contrastive learning benefits from pushing negative pairs farther away rather than pulling positive pairs closer. To solve the above-mentioned problems, we propose a novel spectral GCL framework without positive samples, named SpeGCL. Specifically, to solve the problem that existing GCL methods cannot utilize high-frequency information, SpeGCL uses a Fourier transform to extract high-frequency and low-frequency information of node features, and constructs a contrastive learning mechanism in a Fourier space to obtain better node feature representation. Furthermore, SpeGCL relies entirely on negative samples to refine the graph embedding. We also provide a theoretical justification for the efficacy of using only negative samples in SpeGCL. Extensive experiments on un-supervised learning, transfer learning, and semi-supervised learning have validated the superiority of our SpeGCL framework over the state-of-the-art GCL methods.

Zero-Mean Regularized Spectral Contrastive Learning: Implicitly Mitigating Wrong Connections in Positive-Pair Graphs

Optimal Positive Generation via Latent Transformation for Contrastive Learning

Debiased Graph Contrastive Learning.

Contrastive Visual Feature Filtering for Generalized Zero-Shot Learning

Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss

Tion for negative-free symmetric contrastive learning

Negative as Positive: Enhancing Out-of-distribution Generalization for Graph Contrastive Learning

Zero-CL: Instance and Feature decorrelation for negative-free symmetric contrastive learning.

SpeGCL: Self-supervised Graph Spectrum Contrastive Learning without Positive Samples

Rethinking Spectral Augmentation for Contrast-based Graph Self-Supervised Learning

How Weakly Supervised Information helps Contrastive Learning

On the Importance of Contrastive Loss in Multimodal Learning

A New Mechanism for Eliminating Implicit Conflict in Graph Contrastive Learning

Mining negative samples on contrastive learning via curricular weighting strategy

Debiased Contrastive Learning of Unsupervised Sentence Representations

Negative-Free Self-Supervised Gaussian Embedding of Graphs

Contrastive Learning with Negative Sampling Correction

Contrastive Learning Is Spectral Clustering On Similarity Graph

Graph Self-Contrast Representation Learning

Self-supervised Graph-level Representation Learning with Adversarial Contrastive Learning

P4Contrast: Contrastive Learning with Pairs of Point-Pixel Pairs for RGB-D Scene Understanding