Abstract:As a powerful framework for graph representation learning, Graph Neural Networks (GNNs) have garnered significant attention in recent years. However, to the best of our knowledge, there has been no formal analysis of the logical expressiveness of GNNs as Boolean node classifiers over multi-relational graphs, where each edge carries a specific relation type. In this paper, we investigate $\mathcal{FOC}_2$, a fragment of first-order logic with two variables and counting quantifiers. On the negative side, we demonstrate that the R$^2$-GNN architecture, which extends the local message passing GNN by incorporating global readout, fails to capture $\mathcal{FOC}_2$ classifiers in the general case. Nevertheless, on the positive side, we establish that R$^2$-GNNs models are equivalent to $\mathcal{FOC}_2$ classifiers under certain restricted yet reasonable scenarios. To address the limitations of R$^2$-GNNs regarding expressiveness, we propose a simple graph transformation technique, akin to a preprocessing step, which can be executed in linear time. This transformation enables R$^2$-GNNs to effectively capture any $\mathcal{FOC}_2$ classifiers when applied to the "transformed" input graph. Moreover, we extend our analysis of expressiveness and graph transformation to temporal graphs, exploring several temporal GNN architectures and providing an expressiveness hierarchy for them. To validate our findings, we implement R$^2$-GNNs and the graph transformation technique and conduct empirical tests in node classification tasks against various well-known GNN architectures that support multi-relational or temporal graphs. Our experimental results consistently demonstrate that R$^2$-GNN with the graph transformation outperforms the baseline methods on both synthetic and real-world datasets

Graph Representations for Higher-Order Logic and Theorem Proving

Improving Graph Neural Network Representations of Logical Formulae with Subgraph Pooling

Graph Reasoning Networks

Can Graph Neural Networks Help Logic Reasoning?

Decidability of Graph Neural Networks via Logical Characterizations

Graph Neural Networks for Propositional Model Counting

Logical Expressiveness of Graph Neural Network for Knowledge Graph Reasoning

Efficient Probabilistic Logic Reasoning with Graph Neural Networks

Graph Neural Networks and Boolean Satisfiability

Higher Order Structures For Graph Explanations

Weisfeiler and Leman Go Neural: Higher-Order Graph Neural Networks

Neural-Symbolic Models for Logical Queries on Knowledge Graphs

How Powerful are Graph Neural Networks?

Theory of Graph Neural Networks: Representation and Learning

Provably Powerful Graph Networks

Graph Neural Reasoning for 2-Quantified Boolean Formula Solvers

Calibrate and Boost Logical Expressiveness of GNN Over Multi-Relational and Temporal Graphs

The Logical Expressiveness of Graph Neural Networks

Systematic Reasoning About Relational Domains With Graph Neural Networks

Interpretable Graph Networks Formulate Universal Algebra Conjectures

Logical Distillation of Graph Neural Networks