Commute-Time-Optimised Graphs for GNNs

Igor Sterner,Shiye Su,Petar Veličković

2024-09-05

Abstract:We explore graph rewiring methods that optimise commute time. Recent graph rewiring approaches facilitate long-range interactions in sparse graphs, making such rewirings commute-time-optimal on average. However, when an expert prior exists on which node pairs should or should not interact, a superior rewiring would favour short commute times between these privileged node pairs. We construct two synthetic datasets with known priors reflecting realistic settings, and use these to motivate two bespoke rewiring methods that incorporate the known prior. We investigate the regimes where our rewiring improves test performance on the synthetic datasets. Finally, we perform a case study on a real-world citation graph to investigate the practical implications of our work.

Social and Information Networks,Machine Learning

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to optimize the commute time between nodes in Graph Neural Networks (GNNs) in order to improve the efficiency of information propagation between long - distance nodes. Specifically, the paper focuses on how to optimize the short commute time between specific node pairs through the graph rewiring method in the presence of expert prior knowledge. This is different from most existing graph rewiring methods, which usually aim to optimize the average commute time between all nodes without considering that some node pairs may require more frequent or more direct interactions.

### Main contributions of the paper:

1. **Introduced two new synthetic datasets**: These datasets are used for graph regression tasks and reflect reasonable priors in the real world.
2. **Proposed two non - parametric graph rewiring methods**: Each method is designed for a synthetic dataset and aims to combine the input graph and expert prior to optimize the commute time between specific node pairs.
3. **Conducted extensive experiments**: Evaluated the performance of the proposed graph rewiring methods on synthetic datasets.
4. **Conducted a case study on a real - world citation graph**: Explored the effectiveness of the proposed methods in practical applications.

### Specific problem description:

- **Background**: When Graph Neural Networks (GNNs) process graph - structured data, due to the bottleneck effect (oversquashing) in the graph topology, it is difficult to propagate information between long - distance nodes. Although existing graph rewiring methods can optimize global information propagation, they do not fully utilize expert prior knowledge to optimize the interactions between specific node pairs.
- **Objective**: Through the graph rewiring method, combined with expert prior knowledge, optimize the commute time between specific node pairs, thereby improving the expressive ability and task performance of GNNs.

### Solutions:

- **Synthetic datasets**:
- **Dataset A**: Simulates the protein property prediction task, considering the interactions between neighboring nodes, important node pairs and all other nodes.
- **Dataset B**: Simulates the community interaction task, considering the high interaction requirements between nodes of the same type.

- **Graph rewiring methods**:
- **Dataset A**: Proposed the rewiring methods of aligned - Cayley expander and distance - d - node - pairs.
- **Dataset B**: Proposed the rewiring methods of Cayley - clusters and fully - connected - clusters.

### Experimental results:

- **Dataset A**:
- When the interactions between all nodes are weak (\(c_3 = 0\)), the original graph performs better than any rewiring method.
- When the interactions between all nodes are strong and the interactions of important node pairs are weak (\(c_3\neq0\) and \(c_2 < c_1\)), the Cayley expander and the aligned - Cayley expander perform better than the baseline methods.
- The aligned - Cayley expander performs better in capturing node pairs at distance d, and its advantage gradually weakens as the importance of global interaction increases.

- **Dataset B**:
- When the interactions of neighboring nodes are dominant (low \(c_2/c_1\)), the original graph performs best, and other rewiring methods perform poorly.
- When the interactions of neighboring nodes and color are both important (medium \(c_2/c_1\)), the Cayley expander and Cayley - clusters perform significantly better than the baseline methods.
- When the color interactions are dominant (high \(c_2/c_1\)), Cayley - clusters perform best, followed by the Cayley expander, and the fully - connected method performs worst.

### Case study:

- **ogbn - arxiv**: On a real - world citation graph, using the label - based Cayley - clusters rewiring method, verified the significant performance improvement of this method in extreme cases (perfect prior). Without using perfect prior, by training a simple MLP classifier to predict node colors, faster convergence speed and higher validation accuracy are achieved.

In conclusion, this paper proposes a new graph rewiring method by combining expert prior knowledge, which effectively optimizes the commute time between specific node pairs and improves the performance of GNNs.