Abstract:Graph Neural Networks (GNNs) are powerful in learning semantics of graph data. Recently, a new paradigm "pre-train and prompt" has shown promising results in adapting GNNs to various tasks with less supervised data. The success of such paradigm can be attributed to the more consistent objectives of pre-training and task-oriented prompt tuning, where the pre-trained knowledge can be effectively transferred to downstream tasks. Most existing methods are based on the class prototype vector framework. However, in the few-shot scenarios, given few labeled data, class prototype vectors are difficult to be accurately constructed or learned. Meanwhile, the structure information of graph is usually exploited during pre-training for learning node representations, while neglected in the prompt tuning stage for learning more accurate prototype vectors. In addition, they generally ignore the impact of heterophilous neighborhoods on node representation and are not suitable for heterophilous graphs. To bridge these gaps, we propose a novel pre-training and structure prompt tuning framework for GNNs, namely PSP, which consistently exploits structure information in both pre-training and prompt tuning stages. In particular, PSP 1) employs a dual-view contrastive learning to align the latent semantic spaces of node attributes and graph structure, and 2) incorporates structure information in prompted graph to construct more accurate prototype vectors and elicit more pre-trained knowledge in prompt tuning. We conduct extensive experiments on node classification and graph classification tasks to evaluate the effectiveness of PSP. We show that PSP can lead to superior performance in few-shot scenarios on both homophilous and heterophilous graphs. The implemented code is available at <a class="link-external link-https" href="https://github.com/gqq1210/PSP" rel="external noopener nofollow">this https URL</a>.

Graph Structure Prompt Learning: A Novel Methodology to Improve Performance of Graph Neural Networks

GraphPrompt: Unifying Pre-Training and Downstream Tasks for Graph Neural Networks

A Unified Graph Selective Prompt Learning for Graph Neural Networks

Instance-Aware Graph Prompt Learning

Towards Graph Prompt Learning: A Survey and Beyond

ULTRA-DP: Unifying Graph Pre-training with Multi-task Graph Dual Prompt.

Graph Neural Prompting with Large Language Models

DyGPrompt: Learning Feature and Time Prompts on Dynamic Graphs

Node-Time Conditional Prompt Learning in Dynamic Graphs

G-Prompt: Graphon-based Prompt Tuning for graph classification

Generalized Graph Prompt: Toward a Unification of Pre-Training and Downstream Tasks on Graphs

Reliable and Compact Graph Fine-tuning via GraphSparse Prompting

Self-Pro: A Self-Prompt and Tuning Framework for Graph Neural Networks

All in One: Multi-task Prompting for Graph Neural Networks

All in One: Multi-Task Prompting for Graph Neural Networks (Extended Abstract)

MultiGPrompt for Multi-Task Pre-Training and Prompting on Graphs

PSP: Pre-Training and Structure Prompt Tuning for Graph Neural Networks

SGL-PT: A Strong Graph Learner with Graph Prompt Tuning

ProG: A Graph Prompt Learning Benchmark

Trojan Prompt Attacks on Graph Neural Networks

A Survey on Graph Structure Learning: Progress and Opportunities