Abstract:Although recent advances in higher-order Graph Neural Networks (GNNs) improve the theoretical expressiveness and molecular property predictive performance, they often fall short of the empirical performance of models that explicitly use fragment information as inductive bias. However, for these approaches, there exists no theoretic expressivity study. In this work, we propose the Fragment-WL test, an extension to the well-known Weisfeiler & Leman (WL) test, which enables the theoretic analysis of these fragment-biased GNNs. Building on the insights gained from the Fragment-WL test, we develop a new GNN architecture and a fragmentation with infinite vocabulary that significantly boosts expressiveness. We show the effectiveness of our model on synthetic and real-world data where we outperform all GNNs on Peptides and have 12% lower error than all GNNs on ZINC and 34% lower error than other fragment-biased models. Furthermore, we show that our model exhibits superior generalization capabilities compared to the latest transformer-based architectures, positioning it as a robust solution for a range of molecular modeling tasks.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to bridge the gap between the expressive power and generalization ability of graph neural networks (GNNs) in molecular property prediction tasks. Specifically: 1. **Limitations of expressive power**: - Although existing high - order GNNs theoretically improve their expressive power, in practical applications, their performance is often inferior to those models that explicitly use fragment information as inductive bias. - These fragment - bias models lack theoretical expressive - power analysis, resulting in a mismatch between theory and practical performance. 2. **Challenges of generalization ability**: - Many high - order GNNs perform poorly when dealing with out - of - distribution data and are vulnerable to adversarial attacks and poor generalization problems. - Fragment - bias models are usually only able to handle a single sub - structure and have difficulty comprehensively representing complex molecular structures. To solve these problems, the authors make the following contributions: - **Fragment - WL test**: Extend the classic Weisfeiler - Leman (WL) test to theoretically analyze the expressive power of fragment - bias GNNs. - **New GNN architecture**: Develop a new GNN architecture based on a fragment - based message - passing framework, which significantly improves the expressive power and generalization ability. - **Fragmentation method with infinite vocabulary**: Propose a method to represent molecular graphs using only basic building blocks (such as rings and paths), which can handle an infinite number of sub - structures. - **Experimental verification**: Verify the effectiveness of the new model through synthetic and real - world datasets, demonstrating its superior performance in various molecular modeling tasks, especially in long - distance interaction and generalization ability. These improvements make the new model not only more expressive in theory but also perform well in practical applications, especially when dealing with complex molecular structures and out - of - distribution data. ### Key formulas - **Color update formula for WL test**: \[ c^{(t)}_v=\text{HASH}\left(c^{(t - 1)}_v,\left\{c^{(t - 1)}_w\mid w\in N(v)\right\}\right) \] where \( c^{(t)}_v \) is the color of node \( v \) in the \( t \) - th iteration, and \( N(v) \) is the set of adjacent nodes of node \( v \). - **Color update formula for Fragment - WL test**: \[ X^{\text{NF}}_v = X_v\parallel\lambda\left(\left\{\text{type}(f)\mid v\in f,f\in F\right\}\right) \] where \( X^{\text{NF}}_v \) is the enhanced node feature, \( \lambda \) is an injective function, and \( \parallel \) represents the concatenation operation. Through these improvements, the paper provides a more fine - grained hierarchy of expressive power and proves the superior performance of the new model on multiple benchmark datasets.

Expressivity and Generalization: Fragment-Biases for Molecular GNNs

Towards Bridging Generalization and Expressivity of Graph Neural Networks

Breaking the Expression Bottleneck of Graph Neural Networks

Breaking the Expressive Bottlenecks of Graph Neural Networks

Is fragment-based graph a better graph-based molecular representation for drug design? A comparison study of graph-based models

Will More Expressive Graph Neural Networks do Better on Generative Tasks?

Beyond Weisfeiler-Lehman: A Quantitative Framework for GNN Expressiveness

Fragment-based Pretraining and Finetuning on Molecular Graphs

Utilizing Edge Features in Graph Neural Networks Via Variational Information Maximization

On the Expressive Power of Geometric Graph Neural Networks

Improving Graph Neural Network Expressivity via Subgraph Isomorphism Counting

FragNet: A Graph Neural Network for Molecular Property Prediction with Four Layers of Interpretability

A Complete Expressiveness Hierarchy for Subgraph GNNs via Subgraph Weisfeiler-Lehman Tests

Investigating Graph Neural Networks and Classical Feature-Extraction Techniques in Activity-Cliff and Molecular Property Prediction

Graph Neural Network Expressivity and Meta-Learning for Molecular Property Regression

HiGNN: Hierarchical Informative Graph Neural Networks for Molecular Property Prediction Equipped with Feature-Wise Attention

On the Scalability of GNNs for Molecular Graphs

Gini in a Bottleneck: Sparse Molecular Representations for Graph Convolutional Neural Networks

An Empirical Study of Realized GNN Expressiveness

Expressiveness and Approximation Properties of Graph Neural Networks

Fine-grained Expressivity of Graph Neural Networks