Abstract:Training graph neural networks (GNNs) with good generalizability on large-scale graphs is a challenging problem. Existing methods mainly divide the input graph into multiple subgraphs and train them in different batches to improve training scalability. However, the local batches obtained by such a strategy could contain topological bias compared with the complete graph structure. It has been studied that the topological bias results in more significant gaps between training and testing performances, or worse generalization robustness. A straightforward solution is to utilize contrastive learning, and train node embeddings to be robust and invariant among the augmented imperfect graphs. However, most of the existing work are inefficient by contrasting extensive node pairs at the large-scale graph. With random data augmentation, they may deteriorate the embedding process by transforming well-sampled batches into meaningless graph structures. To bridge the gap between large-scale graph training and contrastive learning, we propose adaptive subgraph contrastive learning (AdaGCL). Given a batch of sampled subgraphs, we propose subgraph-granularity contrastive loss to compare the anchor node with a limited number of subgraphs, which reduces the computation cost. AdaGCL tailors two key components for batch training: (1) Batch-aware view generation to keep the intrinsic individual subgraph structures of batch to learn the informative node embeddings; (2) Batch-aware pair sampling to construct the positive and negative contrasting subgraphs based on anchor node label. Experiments show that AdaGCL can scale up to graphs with millions of nodes, and delivers the consistent improvement than the existing methods on various benchmark datasets. Furthermore, AdaGCL has comparable running time with the state-of-the-art contrastive learning methods that focus on improving efficiency. Finally, ablation studies of the two components of AdaGCL demonstrate their effectiveness to generalize the batch training. The code is in: https://github.com/YL-wang/CIKM_AdaGCL/.

A Subgraph Sampling Method for Training Large-Scale Graph Convolutional Network.

Layer-Dependent Importance Sampling for Training Deep and Large Graph Convolutional Networks

Sampling methods for efficient training of graph convolutional networks: A survey

Resource-Efficient Training for Large Graph Convolutional Networks with Label-Centric Cumulative Sampling

Adaptive Sampling Towards Fast Graph Representation Learning

Edge Convolutional Networks: Decomposing Graph Convolutional Networks for Stochastic Training with Independent Edges

A learnable sampling method for scalable graph neural networks

BNS-GCN: Efficient Full-Graph Training of Graph Convolutional Networks with Partition-Parallelism and Random Boundary Node Sampling

Stochastic Training of Graph Convolutional Networks with Variance Reduction

Learning by Sampling and Compressing: Efficient Graph Representation Learning with Extremely Limited Annotations

CDGCN: an Effective and Efficient Algorithm Based on Community Detection for Training Deep and Large Graph Convolutional Networks

GraphSAINT: Graph Sampling Based Inductive Learning Method

Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks

MG-GCN: Fast and Effective Learning with Mix-grained Aggregators for Training Large Graph Convolutional Networks

PPSGCN: A Privacy-Preserving Subgraph Sampling Based Distributed GCN Training Method

Accurate, Efficient and Scalable Graph Embedding

Non-recursive graph convolutional networks

Clustering with Entropy-based Recombination for Training GCNs on Large Graphs

Contrastive Graph Convolutional Networks with Generative Adjacency Matrix

AdaGCL: Adaptive Subgraph Contrastive Learning to Generalize Large-scale Graph Training