Abstract:Graph-structured data is ubiquitous in the world which models complexrelationships between objects, enabling various Web applications. Dailyinfluxes of unlabeled graph data on the Web offer immense potential for theseapplications. Graph self-supervised algorithms have achieved significantsuccess in acquiring generic knowledge from abundant unlabeled graph data.These pre-trained models can be applied to various downstream Web applications,saving training time and improving downstream (target) performance. However,different graphs, even across seemingly similar domains, can differsignificantly in terms of attribute semantics, posing difficulties, if notinfeasibility, for transferring the pre-trained models to downstream tasks.Concretely speaking, for example, the additional task-specific node informationin downstream tasks (specificity) is usually deliberately omitted so that thepre-trained representation (transferability) can be leveraged. The trade-off assuch is termed as "transferability-specificity dilemma" in this work. Toaddress this challenge, we introduce an innovative deployment module coined asGraphControl, motivated by ControlNet, to realize better graph domain transferlearning. Specifically, by leveraging universal structural pre-trained modelsand GraphControl, we align the input space across various graphs andincorporate unique characteristics of target data as conditional inputs. Theseconditions will be progressively integrated into the model during fine-tuningor prompt tuning through ControlNet, facilitating personalized deployment.Extensive experiments show that our method significantly enhances theadaptability of pre-trained models on target attributed datasets, achieving1.4-3x performance gain. Furthermore, it outperforms training-from-scratchmethods on target data with a comparable margin and exhibits fasterconvergence.

Unsupervisedly Learned Latent Graphs as Transferable Representations

GLoMo: Unsupervised Learning of Transferable Relational Graphs

GraphControl: Adding Conditional Control to Universal Graph Pre-trained Models for Graph Domain Transfer Learning

Learning the Implicit Semantic Representation on Graph-Structured Data

UniGraph: Learning a Unified Cross-Domain Foundation Model for Text-Attributed Graphs

Deep Feature Learning for Graphs

Transfer Active Learning For Graph Neural Networks

Intrinsic Geometric Information Transfer Learning on Multiple Graph-Structured Datasets

Unsupervised Inductive Graph-Level Representation Learning Via Graph-Graph Proximity

Learning graph-level representation from local-structural distribution with Graph Neural Networks

Learning Representations without Compositional Assumptions

GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graph

Unsupervised Inductive Whole-Graph Embedding by Preserving Graph Proximity

Adaptive Transfer Learning on Graph Neural Networks

Learning Deep Graph Representations via Convolutional Neural Networks

There is more to graphs than meets the eye: Learning universal features with self-supervision

Transfer Learning with Graph Co-Regularization.

GraphTER: Unsupervised Learning of Graph Transformation Equivariant Representations via Auto-Encoding Node-Wise Transformations

Transferability Properties of Graph Neural Networks

Towards Understanding the Transferability of Deep Representations

Simple Unsupervised Graph Representation Learning