Improving Expressivity of GNNs with Subgraph-specific Factor Embedded Normalization

Kaixuan Chen,Shunyu Liu,Tongtian Zhu,Tongya Zheng,Haofei Zhang,Zunlei Feng,Jingwen Ye,Mingli Song

2023-06-12

Abstract:Graph Neural Networks (GNNs) have emerged as a powerful category of learning architecture for handling graph-structured data. However, existing GNNs typically ignore crucial structural characteristics in node-induced subgraphs, which thus limits their expressiveness for various downstream tasks. In this paper, we strive to strengthen the representative capabilities of GNNs by devising a dedicated plug-and-play normalization scheme, termed as SUbgraph-sPEcific FactoR Embedded Normalization (SuperNorm), that explicitly considers the intra-connection information within each node-induced subgraph. To this end, we embed the subgraph-specific factor at the beginning and the end of the standard BatchNorm, as well as incorporate graph instance-specific statistics for improved distinguishable capabilities. In the meantime, we provide theoretical analysis to support that, with the elaborated SuperNorm, an arbitrary GNN is at least as powerful as the 1-WL test in distinguishing non-isomorphism graphs. Furthermore, the proposed SuperNorm scheme is also demonstrated to alleviate the over-smoothing phenomenon. Experimental results related to predictions of graph, node, and link properties on the eight popular datasets demonstrate the effectiveness of the proposed method. The code is available at <a class="link-external link-https" href="https://github.com/chenchkx/SuperNorm" rel="external noopener nofollow">this https URL</a>.

Machine Learning,Artificial Intelligence

What problem does this paper attempt to address?

This paper attempts to solve two main problems faced by Graph Neural Networks (GNNs) when processing graph - structured data:

1. **Lack of Expressive Power**: Existing GNNs usually overlook the key structural features in node - induced subgraphs, which limits their expressive power in various downstream tasks. Specifically, the expressive power of existing GNNs is at most as powerful as the Weisfeiler - Lehman (1 - WL) test and cannot distinguish non - isomorphic graphs, especially when dealing with sub - tree isomorphic subgraphs.

2. **Over - smoothing Problem**: As the number of GNN layers increases, node representations become more and more similar, resulting in information loss and making it difficult to distinguish nodes of different classes, thus affecting the performance of GNNs on node and link prediction tasks.

To solve these problems, the authors propose a new normalization framework - SuperNorm (SUbgraph - sPEcific Facto REmbedded Normalization), which enhances the expressive power of GNNs and alleviates the over - smoothing problem by embedding subgraph - specific factors. Specifically, SuperNorm improves GNNs in the following ways:

- **Introducing Subgraph - specific Factors**: Calculate the unique identifier of the subgraph through a hash function to ensure that each non - isomorphic subgraph has a unique representation.

- **Enhancing the Normalization Scheme**: Embed subgraph - specific factors into standard BatchNorm and combine graph - instance - specific statistical information to improve the discrimination ability.

- **Supported by Theoretical Analysis**: Provide theoretical analysis to prove that SuperNorm can make any GNN at least as powerful as the 1 - WL test and can alleviate the over - smoothing phenomenon.

Experimental results show that SuperNorm significantly improves the performance of GNNs in graph, node, and link property prediction tasks on multiple popular graph datasets.