Abstract: Self-supervised learning on graphs has recently achieved remarkable success in graph representation learning. With hundreds of self-supervised pretext tasks proposed over the past few years, the research community has greatly developed, and the key is no longer to design more powerful but complex pretext tasks, but to make more effective use of those already on hand. This paper studies the problem of how to automatically, adaptively, and dynamically learn instance-level self-supervised learning strategies for each node from a given pool of pretext tasks. In this paper, we propose a novel multi-teacher knowledge distillation framework for Automated Graph Self-Supervised Learning (AGSSL), which consists of two main branches: (i) Knowledge Extraction: training multiple teachers with different pretext tasks, so as to extract different levels of knowledge with different inductive biases; (ii) Knowledge Integration: integrating different levels of knowledge and distilling them into the student model. Without simply treating different teachers as equally important, we provide a provable theoretical guideline for how to integrate the knowledge of different teachers, i.e., the integrated teacher probability should be close to the true Bayesian class-probability. To approach the theoretical optimum in practice, two adaptive knowledge integration strategies are proposed to construct a relatively "good" integrated teacher. Extensive experiments on eight datasets show that AGSSL can benefit from multiple pretext tasks, outperforming the corresponding individual tasks; by combining a few simple but classical pretext tasks, the resulting performance is comparable to other leading counterparts.

Interactive Graph Construction for Graph-Based Semi-Supervised Learning.

Self-supervised Learning on Graphs: Contrastive, Generative,or Predictive

Large Margin Graph Construction For Semi-Supervised Learning

Learning safe graph construction from multiple graphs

Graph-based Semi-supervised Learning: A Comprehensive Review

Semi-supervised Classification Via Low Rank Graph

Semi-supervised Learning via Bipartite Graph Construction with Adaptive Neighbors

Automated Graph Self-supervised Learning via Multi-teacher Knowledge Distillation

Label Information Guided Graph Construction for Semi-Supervised Learning

Image colourisation using graph-based semi-supervised learning

Semi-supervised learning via sparse model

Efficient And Robust Semi-Supervised Learning Over A Sparse-Regularized Graph

Graph Quality Judgement: A Large Margin Expedition.

Semi-Supervised Learning with Close-Form Label Propagation Using a Bipartite Graph

Robust Self-Tuning Semi-Supervised Learning.

Scalable Graph-Based Semi-Supervised Learning Through Sparse Bayesian Model.

Probabilistic Semi-Supervised Learning via Sparse Graph Structure Learning

Semi-supervised Active Learning for Semi-supervised Models: Exploit Adversarial Examples with Graph-based Virtual Labels

Semi-supervised Learning for Convolutional Neural Networks via Online Graph Construction

Active Model Selection for Graph-Based Semi-Supervised Learning

Label Diagnosis Through Self Tuning for Web Image Search