Abstract:Contrastive learning is a significant paradigm in graph self-supervised learning. However, it requires negative samples to prevent model collapse and learn discriminative representations. These negative samples inevitably lead to heavy computation, memory overhead and class collision, compromising the representation learning. Recent studies present that methods obviating negative samples can attain competitive performance and scalability enhancements, exemplified by bootstrapped graph latents (BGRL). However, BGRL neglects the inherent graph homophily, which provides valuable insights into underlying positive pairs. Our motivation arises from the observation that subtly introducing a few ground-truth positive pairs significantly improves BGRL. Although we can't obtain ground-truth positive pairs without labels under the self-supervised setting, edges in the graph can reflect noisy positive pairs, i.e., neighboring nodes often share the same label. Therefore, we propose to expand the positive pair set with node-neighbor pairs. Subsequently, we introduce a cross-attention module to predict the supportiveness score of a neighbor with respect to the anchor node. This score quantifies the positive support from each neighboring node, and is encoded into the training objective. Consequently, our method mitigates class collision from negative and noisy positive samples, concurrently enhancing intra-class compactness. Extensive experiments are conducted on five benchmark datasets and three downstream task node classification, node clustering, and node similarity search. The results demonstrate that our method generates node representations with enhanced intra-class compactness and achieves state-of-the-art performance.

Hyperspherically Regularized Networks for Self-Supervision

Hyperspherical Consistency Regularization.

Bootstrap your own latent: A new approach to self-supervised Learning

Dehaze on Small-Scale Datasets Via Self-Supervised Learning

ROBYOL: Random-Occlusion-Based BYOL for Hyperspectral Image Classification

Adaptive Similarity Bootstrapping for Self-Distillation based Representation Learning

Learning Where to Learn in Cross-View Self-Supervised Learning

Learning with Hyperspherical Uniformity

Understanding self-supervised Learning Dynamics without Contrastive Pairs

The Edge of Orthogonality: A Simple View of What Makes BYOL Tick

Consensus Clustering With Unsupervised Representation Learning

The Common Stability Mechanism behind most Self-Supervised Learning Approaches

Nearest Neighboring Self-Supervised Learning for Hyperspectral Image Classification

Run Away From your Teacher: Understanding BYOL by a Novel Self-Supervised Approach

Bootstrap Latents of Nodes and Neighbors for Graph Self-Supervised Learning

Compressive Visual Representations

Exploring Self-Supervised Regularization for Supervised and Semi-Supervised Learning

Hierarchically Decoupled Spatial-Temporal Contrast for Self-supervised Video Representation Learning

Hyperbolic Self-supervised Contrastive Learning Based Network Anomaly Detection

Hyperbolic Contrastive Learning for Visual Representations beyond Objects

VISER: Visual Self-Regularization