Abstract:Despite the promising representation learning of graph neural networks (GNNs), the supervised training of GNNs notoriously requires large amounts of labeled data from each application. An effective solution is to apply the transfer learning in graph: using easily accessible information to pre-train GNNs, and fine-tuning them to optimize the downstream task with only a few labels. Recently, many efforts have been paid to design the self-supervised pretext tasks, and encode the universal graph knowledge among the various applications. However, they rarely notice the inherent training objective gap between the pretext and downstream tasks. This significant gap often requires costly fine-tuning for adapting the pre-trained model to downstream problem, which prevents the efficient elicitation of pre-trained knowledge and then results in poor results. Even worse, the naive pre-training strategy usually deteriorates the downstream task, and damages the reliability of transfer learning in graph data. To bridge the task gap, we propose a novel transfer learning paradigm to generalize GNNs, namely graph pre-training and prompt tuning (GPPT). Specifically, we first adopt the masked edge prediction, the most simplest and popular pretext task, to pre-train GNNs. Based on the pre-trained model, we propose the graph prompting function to modify the standalone node into a token pair, and reformulate the downstream node classification looking the same as edge prediction. The token pair is consisted of candidate label class and node entity. Therefore, the pre-trained GNNs could be applied without tedious fine-tuning to evaluate the linking probability of token pair, and produce the node classification decision. The extensive experiments on eight benchmark datasets demonstrate the superiority of GPPT, delivering an average improvement of 4.29% in few-shot graph analysis and accelerating the model convergence up to 4.32X. The code is available in: https://github.com/MingChen-Sun/GPPT.

GraphControl: Adding Conditional Control to Universal Graph Pre-trained Models for Graph Domain Transfer Learning

Adaptive Scheduling for Internet of Vehicles Using Deconfounded Graph Transfer Learning

GraphCLIP: Enhancing Transferability in Graph Foundation Models for Text-Attributed Graphs

Adaptive Transfer Learning on Graph Neural Networks

Can Modifying Data Address Graph Domain Adaptation?

ULTRA-DP: Unifying Graph Pre-training with Multi-task Graph Dual Prompt.

Augmenting Knowledge Transfer across Graphs

Explaining and Adapting Graph Conditional Shift

Core Knowledge Learning Framework for Graph Adaptation and Scalability Learning

All in One and One for All: A Simple yet Effective Method towards Cross-domain Graph Pretraining

GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training

Graph Transfer Learning via Adversarial Domain Adaptation with Graph Convolution

GM-transfer: Graph-based model for transfer learning

Transfer Learning with Graph Co-Regularization.

UniGraph: Learning a Unified Cross-Domain Foundation Model for Text-Attributed Graphs

Transfer Active Learning For Graph Neural Networks

Strategies for Pre-training Graph Neural Networks

GPL-GNN: Graph Prompt Learning for Graph Neural Network

Scalable Multi-Source Pre-training for Graph Neural Networks

GPPT: Graph Pre-training and Prompt Tuning to Generalize Graph Neural Networks

Graph-Relational Domain Adaptation