Abstract:Graph Neural Networks (GNNs) are an effective framework for representation learning of graphs. GNNs follow a neighborhood aggregation scheme, where the representation vector of a node is computed by recursively aggregating and transforming representation vectors of its neighboring nodes. Many GNN variants have been proposed and have achieved state-of-the-art results on both node and graph classification tasks. However, despite GNNs revolutionizing graph representation learning, there is limited understanding of their representational properties and limitations. Here, we present a theoretical framework for analyzing the expressive power of GNNs to capture different graph structures. Our results characterize the discriminative power of popular GNN variants, such as Graph Convolutional Networks and GraphSAGE, and show that they cannot learn to distinguish certain simple graph structures. We then develop a simple architecture that is provably the most expressive among the class of GNNs and is as powerful as the Weisfeiler-Lehman graph isomorphism test. We empirically validate our theoretical findings on a number of graph classification benchmarks, and demonstrate that our model achieves state-of-the-art performance.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is the expressiveness and limitations of Graph Neural Networks (GNNs) in representation learning. Specifically, the authors proposed a theoretical framework to analyze the expressiveness of GNNs in capturing different graph structures. They studied popular GNN variants, such as Graph Convolutional Networks (GCNs) and GraphSAGE, and found that these models are unable to distinguish certain simple graph structures. Based on this, the authors developed a new neural architecture - Graph Isomorphism Network (GIN), which has been proven to be one of the most expressive GNNs, and its expressiveness is equivalent to the Weisfeiler - Lehman graph isomorphism test. In addition, the authors experimentally demonstrated their theoretical findings and showed that the performance of GIN on multiple graph classification benchmarks has reached the state - of - the - art level. ### Main Contributions 1. **Theoretical Analysis of Expressiveness**: The paper formally defined how to evaluate the expressiveness of different GNN variants in learning representations and distinguishing different graph structures. 2. **Condition Establishment**: Established the conditions for neighbor aggregation and graph read - out functions, enabling GNNs to achieve the same expressiveness as the Weisfeiler - Lehman test. 3. **Indistinguishable Graph Structures**: Identified the graph structures that popular GNN variants (such as GCN and GraphSAGE) cannot distinguish, and precisely described the types of graph structures that these models can capture. 4. **New Architecture Development**: Developed the Graph Isomorphism Network (GIN) and proved that its expressiveness is the same as that of the Weisfeiler - Lehman test. 5. **Experimental Verification**: Verified the theoretical findings through experiments and demonstrated the superior performance of GIN in graph classification tasks. ### Theoretical Framework The theoretical framework proposed in the paper is mainly based on the concept of multiset. The neighbor aggregation of GNN can be regarded as an aggregation function for multiset. In order to have strong expressiveness, GNN must be able to aggregate different multisets into different representations. The authors theoretically characterized the expressiveness of GNN by analyzing different multiset functions and their discrimination abilities. ### Experimental Results The authors verified the theoretical results through experiments on multiple graph classification datasets. The experiments showed that GIN not only has the highest expressiveness theoretically, but also performs excellently in practical applications, especially in fitting the training data almost perfectly, while other GNN variants with weaker expressiveness are often severely under - fitting the training data. In addition, GNNs with stronger expressiveness also outperform other models in test - set accuracy and have reached the state - of - the - art performance on multiple graph classification benchmarks. ### Conclusion This paper proved the key issues regarding the expressiveness and limitations of GNNs through theoretical analysis and experiments, and proposed a new GNN architecture - GIN, which has shown excellent performance both theoretically and practically. These research results provide a theoretical basis and practical guidance for designing more powerful GNNs.

How Powerful are Graph Neural Networks?

On the Expressive Power of Graph Neural Networks

Discovering the Representation Bottleneck of Graph Neural Networks from Multi-order Interactions

Are Powerful Graph Neural Nets Necessary? A Dissection on Graph Classification

The Expressive Power of Graph Neural Networks: A Survey

Rethinking the Expressive Power of GNNs via Graph Biconnectivity

How Powerful are Spectral Graph Neural Networks.

Representation Power of Graph Neural Networks: Improved Expressivity via Algebraic Analysis

GOING BEYOND 1-WL EXPRESSIVE POWER WITH 1-LAYER GRAPH NEURAL NETWORKS

Analyzing the Expressive Power of Graph Neural Networks in a Spectral Perspective

Discovering the Representation Bottleneck of Graph Neural Networks

How does over-squashing affect the power of GNNs?

Improving Graph Neural Network Expressivity via Subgraph Isomorphism Counting

On Graph Neural Networks versus Graph-Augmented MLPs

Graph Neural Networks Exponentially Lose Expressive Power for Node Classification

Expressivity of Graph Neural Networks Through the Lens of Adversarial Robustness

Provably Powerful Graph Networks

What Do GNNs Actually Learn? Towards Understanding their Representations

The Expressive Power of Graph Neural Networks as a Query Language

Revisiting Graph Neural Networks on Graph-level Tasks: Comprehensive Experiments, Analysis, and Improvements