Abstract:Augmentation techniques and sampling strategies are crucial in contrastive learning, but in most existing works, augmentation techniques require careful design, and their sampling strategies can only capture a small amount of intrinsic supervision information. Additionally, the existing methods require complex designs to obtain two different representations of the data. To overcome these limitations, we propose a novel framework called the Self-Contrastive Graph Diffusion Network (SCGDN). Our framework consists of two main components: the Attentional Module (AttM) and the Diffusion Module (DiFM). AttM aggregates higher-order structure and feature information to get an excellent embedding, while DiFM balances the state of each node in the graph through Laplacian diffusion learning and allows the cooperative evolution of adjacency and feature information in the graph. Unlike existing methodologies, SCGDN is an augmentation-free approach that avoids "sampling bias" and semantic drift, without the need for pre-training. We conduct a high-quality sampling of samples based on structure and feature information. If two nodes are neighbors, they are considered positive samples of each other. If two disconnected nodes are also unrelated on $k$NN graph, they are considered negative samples for each other. The contrastive objective reasonably uses our proposed sampling strategies, and the redundancy reduction term minimizes redundant information in the embedding and can well retain more discriminative information. In this novel framework, the graph self-contrastive learning paradigm gives expression to a powerful force. SCGDN effectively balances between preserving high-order structure information and avoiding overfitting. The results manifest that SCGDN can consistently generate outperformance over both the contrastive methods and the classical methods.

Iterative Graph Self-Distillation

DiscoGNN: A Sample-Efficient Framework for Self-Supervised Graph Representation Learning

Automated Graph Self-supervised Learning via Multi-teacher Knowledge Distillation

Does Graph Distillation See Like Vision Dataset Counterpart?

Distilling Knowledge from Self-Supervised Teacher by Embedding Graph Alignment

Teaching Yourself: Graph Self-Distillation on Neighborhood for Node Classification

Tolerant Self-Distillation for Image Classification

On Self-Distilling Graph Neural Network

Bag of Instances Aggregation Boosts Self-supervised Distillation

Dual teachers for self-knowledge distillation

Restructuring the Teacher and Student in Self-Distillation

Scalable Consistency Training for Graph Neural Networks via Self-Ensemble Self-Distillation

Self-Contrastive Graph Diffusion Network

Confidence-aware Self-Semantic Distillation on Knowledge Graph Embedding

A Teacher-Free Graph Knowledge Distillation Framework with Dual Self-Distillation

Cross-Level Multi-Instance Distillation for Self-Supervised Fine-Grained Visual Categorization

Distilling a Powerful Student Model via Online Knowledge Distillation

SD-DiT: Unleashing the Power of Self-supervised Discrimination in Diffusion Transformer

Improving Structural and Semantic Global Knowledge in Graph Contrastive Learning with Distillation

Enhancing Graph Self-Supervised Learning with Graph Interplay

GEIKD : Self-knowledge distillation based on gated ensemble networks and influences-based label noise removal