Abstract:Positional encodings (PEs) are essential for building powerful and expressive graph neural networks and graph transformers, as they effectively capture the relative spatial relationships between nodes. Although extensive research has been devoted to PEs in undirected graphs, PEs for directed graphs remain relatively unexplored. This work seeks to address this gap. We first introduce the notion of Walk Profile, a generalization of walk-counting sequences for directed graphs. A walk profile encompasses numerous structural features crucial for directed graph-relevant applications, such as program analysis and circuit performance prediction. We identify the limitations of existing PE methods in representing walk profiles and propose a novel Multi-q Magnetic Laplacian PE, which extends the Magnetic Laplacian eigenvector-based PE by incorporating multiple potential factors. The new PE can provably express walk profiles. Furthermore, we generalize prior basis-invariant neural networks to enable the stable use of the new PE in the complex domain. Our numerical experiments validate the expressiveness of the proposed PEs and demonstrate their effectiveness in solving sorting network satisfiability and performing well on general circuit benchmarks. Our code is available at <a class="link-external link-https" href="https://github.com/Graph-COM/Multi-q-Maglap" rel="external noopener nofollow">this https URL</a>.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to design effective positional encodings (PEs) for directed graphs to capture the bidirectional relationships between nodes. Although positional encodings have been widely studied and achieved remarkable results in undirected graphs, positional encodings in directed graphs are still relatively unexplored. Specifically, the paper aims to:
1. **Define and introduce new concepts**: Propose a new concept named "Walk Profile", which generalizes the walk - count sequence in undirected graphs and can represent many important bidirectional relationships in directed graphs, such as directed shortest - path distances, common successor/predecessor nodes, and feed - forward loops, etc.
2. **Identify the limitations of existing methods**: Point out the deficiencies of existing positional encoding methods (such as symmetric Laplacian PE, SVD - PE, and magnetic Laplacian PE) in representing walk profiles and their inability to effectively capture bidirectional relationships in directed graphs.
3. **Propose a new solution**: To overcome these limitations, the paper proposes a new Multi - q Magnetic Laplacian PE. By combining the eigenvalues and eigenvectors of magnetic Laplacian matrices with multiple different potential constants \(q\), the new method can effectively express walk profiles and can completely reconstruct the number of these walks through Fourier transform.
4. **Ensure stability and invariance**: Introduce the first basis - invariant and stable neural architecture to handle complex eigenvectors, thereby solving the problems of fundamental ambiguity and instability in existing methods.
5. **Verify effectiveness**: Verify the effectiveness of the proposed Multi - q Magnetic Laplacian PE through numerical experiments, demonstrating its superior performance in solving tasks such as sorting - network satisfiability and circuit - performance prediction.
In summary, this paper is committed to filling the research gap in the field of positional encoding for directed graphs, proposing more powerful and expressive positional encoding methods, thereby enhancing the capabilities of graph neural networks and graph transformers in handling tasks related to directed graphs.
### Formula Examples
- Definition of Walk Profile:
\[
\Phi_{u,v}(\ell, k)=\text{The number of bidirectional walks from node }u\text{ to node }v\text{ with length }\ell\text{ that contain exactly }k\text{ forward edges and }\ell - k\text{ reverse edges}
\]
- Magnetic Laplacian Matrix:
\[
[A_q]_{u,v}=\exp\left(i2\pi q(A_{u,v}-A_{v,u})\right)\quad\text{if }(u, v)\in E\text{ or }(v, u)\in E
\]
\[
L_q = I - D^{-1/2}A_qD^{-1/2}
\]
- Multi - q Magnetic Laplacian Positional Encoding:
\[
z^{\vec{q}}_u=([V_{q_1}]_{u,:},\ldots,[V_{q_Q}]_{u,:})^\top
\]
where \(V_{q_i}\) is the eigenvector of \(L_{q_i}\).
These formulas show how to capture and represent complex structural information in directed graphs through mathematical tools.