Abstract:Augmentation techniques and sampling strategies are crucial in contrastive learning, but in most existing works, augmentation techniques require careful design, and their sampling strategies can only capture a small amount of intrinsic supervision information. Additionally, the existing methods require complex designs to obtain two different representations of the data. To overcome these limitations, we propose a novel framework called the Self-Contrastive Graph Diffusion Network (SCGDN). Our framework consists of two main components: the Attentional Module (AttM) and the Diffusion Module (DiFM). AttM aggregates higher-order structure and feature information to get an excellent embedding, while DiFM balances the state of each node in the graph through Laplacian diffusion learning and allows the cooperative evolution of adjacency and feature information in the graph. Unlike existing methodologies, SCGDN is an augmentation-free approach that avoids "sampling bias" and semantic drift, without the need for pre-training. We conduct a high-quality sampling of samples based on structure and feature information. If two nodes are neighbors, they are considered positive samples of each other. If two disconnected nodes are also unrelated on $k$NN graph, they are considered negative samples for each other. The contrastive objective reasonably uses our proposed sampling strategies, and the redundancy reduction term minimizes redundant information in the embedding and can well retain more discriminative information. In this novel framework, the graph self-contrastive learning paradigm gives expression to a powerful force. SCGDN effectively balances between preserving high-order structure information and avoiding overfitting. The results manifest that SCGDN can consistently generate outperformance over both the contrastive methods and the classical methods.

ProSampler: Improving Contrastive Learning by Better Mini-batch Sampling

BatchSampler: Sampling Mini-Batches for Contrastive Learning in Vision, Language, and Graphs

Optimal Positive Generation via Latent Transformation for Contrastive Learning

ProGCL: Rethinking Hard Negative Mining in Graph Contrastive Learning

Debiased Graph Contrastive Learning.

$\mathbb{X}$-Sample Contrastive Loss: Improving Contrastive Learning with Sample Similarity Graphs

Global Contrastive Batch Sampling via Optimization on Sample Permutations

Select Your Own Counterparts: Self-Supervised Graph Contrastive Learning With Positive Sampling

Synthetic Hard Negative Samples for Contrastive Learning

Rethinking Samples Selection for Contrastive Learning: Mining of Potential Samples

Contrastive Learning with Negative Sampling Correction

Provable Stochastic Optimization for Global Contrastive Learning: Small Batch Does Not Harm Performance.

Boosting Graph Contrastive Learning Via Adaptive Sampling

Self-Contrastive Graph Diffusion Network

Siamese Prototypical Contrastive Learning

When hard negative sampling meets supervised contrastive learning

P4Contrast: Contrastive Learning with Pairs of Point-Pixel Pairs for RGB-D Scene Understanding

From Pretext to Purpose: Batch-Adaptive Self-Supervised Learning

Prototypical Graph Contrastive Learning

Graph Self-Contrast Representation Learning

Asymmetric Patch Sampling for Contrastive Learning