Abstract:Graph representation learning has attracted lots of attention recently. Existing graph neural networks fed with the complete graph data are not scalable due to limited computation and memory costs. Thus, it remains a great challenge to capture rich information in large-scale graph data. Besides, these methods mainly focus on supervised learning and highly depend on node label information, which is expensive to obtain in the real world. As to unsupervised network embedding approaches, they overemphasize node proximity instead, whose learned representations can hardly be used in downstream application tasks directly. In recent years, emerging self-supervised learning provides a potential solution to address the aforementioned problems. However, existing self-supervised works also operate on the complete graph data and are biased to fit either global or very local (1-hop neighborhood) graph structures in defining the mutual information based loss terms. In this paper, a novel self-supervised representation learning method via Sub-graph Contrast, namely SUBG-CON, is proposed by utilizing the strong correlation between central nodes and their sampled subgraphs to capture regional structure information. Instead of learning on the complete input graph data, with a novel data augmentation strategy, SUBG-CON learns node representations through a contrastive loss defined based on subgraphs sampled from the original graph instead. Compared with existing graph representation learning approaches, SUBG-CON has prominent performance advantages in weaker supervision requirements, model learning scalability, and parallelization. Extensive experiments verify both the effectiveness and the efficiency of our work compared with both classic and state-of-the-art graph representation learning approaches on multiple real world large-scale benchmark datasets from different domains.

There is more to graphs than meets the eye: Learning universal features with self-supervision

Homophily-Enhanced Self-Supervision for Graph Structure Learning: Insights and Directions.

Self-supervised Learning on Graphs: Contrastive, Generative,or Predictive

Self-supervised Graph Representations with Generative Adversarial Learning

Self-supervision meets kernel graph neural models: From architecture to augmentations

Feature propagation as self-supervision signals on graphs

GraphControl: Adding Conditional Control to Universal Graph Pre-trained Models for Graph Domain Transfer Learning

Decoupled Self-supervised Learning for Non-Homophilous Graphs

Unsupervisedly Learned Latent Graphs as Transferable Representations

Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Prediction

Self-supervised graph representation learning using multi-scale subgraph views contrast

Multi-Scale Contrastive Siamese Networks for Self-Supervised Graph Representation Learning

Towards Semi-supervised Universal Graph Classification

UniGraph: Learning a Unified Cross-Domain Foundation Model for Text-Attributed Graphs

Self-supervised Training of Graph Convolutional Networks

Generalized Graph Embedding Models

From Canonical Correlation Analysis to Self-supervised Graph Neural Networks

Alleviating neighbor bias: augmenting graph self-supervise learning with structural equivalent positive samples

AnyGraph: Graph Foundation Model in the Wild

Sub-graph Contrast for Scalable Self-Supervised Graph Representation Learning

Better and Faster: Knowledge Transfer from Multiple Self-supervised Learning Tasks via Graph Distillation for Video Classification