GraphAny: A Foundation Model for Node Classification on Any Graph

Jianan Zhao,Hesham Mostafa,Mikhail Galkin,Michael Bronstein,Zhaocheng Zhu,Jian Tang

2024-06-03

Abstract:Foundation models that can perform inference on any new task without requiring specific training have revolutionized machine learning in vision and language applications. However, applications involving graph-structured data remain a tough nut for foundation models, due to challenges in the unique feature- and label spaces associated with each graph. Traditional graph ML models such as graph neural networks (GNNs) trained on graphs cannot perform inference on a new graph with feature and label spaces different from the training ones. Furthermore, existing models learn functions specific to the training graph and cannot generalize to new graphs. In this work, we tackle these two challenges with a new foundational architecture for inductive node classification named GraphAny. GraphAny models inference on a new graph as an analytical solution to a LinearGNN, thereby solving the first challenge. To solve the second challenge, we learn attention scores for each node to fuse the predictions of multiple LinearGNNs. Specifically, the attention module is carefully parameterized as a function of the entropy-normalized distance-features between multiple LinearGNNs predictions to ensure generalization to new graphs. Empirically, GraphAny trained on the Wisconsin dataset with only 120 labeled nodes can effectively generalize to 30 new graphs with an average accuracy of 67.26\% in an inductive manner, surpassing GCN and GAT trained in the supervised regime, as well as other inductive baselines.

Machine Learning,Social and Information Networks

What problem does this paper attempt to address?

The paper aims to address two main challenges faced by Graph Neural Networks (GNNs) when dealing with new graphs:

1. **Differences in feature and label spaces**: Existing GNN models can only perform inference on the feature and label spaces encountered during training and cannot handle feature and label spaces different from the training data.
2. **Ability to generalize to new graphs**: Existing models learn functions specific to the training graphs and cannot directly generalize to new graph structures.

To address these two issues, the paper proposes the GraphAny architecture, which can perform node classification tasks on any new graph. Specifically, GraphAny consists of two parts:

- **LinearGNN**: Performs inference on new feature and label spaces without the need for a training step.
- **Attention mechanism**: Uses entropy-normalized distance features to fuse the predictions of multiple LinearGNNs, ensuring the model can generalize to new graphs.

Experimental results show that after training on a single dataset, GraphAny can effectively generalize to 30 new graphs with different feature and label spaces, outperforming baseline methods such as supervised GCN and GAT. This indicates that GraphAny has significant advantages in addressing the problem of cross-graph generalization.