PANDA: Expanded Width-Aware Message Passing Beyond Rewiring

Jeongwhan Choi,Sumin Park,Hyowon Wi,Sung-Bae Cho,Noseong Park

2024-07-20

Abstract:Recent research in the field of graph neural network (GNN) has identified a critical issue known as "over-squashing," resulting from the bottleneck phenomenon in graph structures, which impedes the propagation of long-range information. Prior works have proposed a variety of graph rewiring concepts that aim at optimizing the spatial or spectral properties of graphs to promote the signal propagation. However, such approaches inevitably deteriorate the original graph topology, which may lead to a distortion of information flow. To address this, we introduce an expanded width-aware (PANDA) message passing, a new message passing paradigm where nodes with high centrality, a potential source of over-squashing, are selectively expanded in width to encapsulate the growing influx of signals from distant nodes. Experimental results show that our method outperforms existing rewiring methods, suggesting that selectively expanding the hidden state of nodes can be a compelling alternative to graph rewiring for addressing the over-squashing.

Machine Learning,Artificial Intelligence

What problem does this paper attempt to address?

This paper attempts to address the "over-squashing" problem in Graph Neural Networks (GNNs). Specifically, when GNNs handle long-distance dependencies, the aggregation of information from a large number of neighboring nodes into a fixed-size feature vector can lead to information loss, thereby reducing the network's expressive power and affecting model performance. To solve this problem, some existing methods optimize the spatial or spectral properties of the graph through rewiring to facilitate signal propagation. However, these methods inevitably disrupt the original topology of the graph, leading to information distortion.

The paper proposes a new message-passing framework called PANDA (Expanded Width-Aware Message Passing). This framework alleviates the over-squashing problem by selectively expanding the width of nodes with high centrality (i.e., potential bottleneck nodes) rather than changing the graph's connectivity. Experimental results show that the PANDA method outperforms existing rewiring methods on various benchmark datasets and maintains higher feature sensitivity, performing well even in deeper layers. Additionally, PANDA can sustain stable signal propagation under high effective resistance, addressing the over-smoothing issue that rewiring methods might cause.