Abstract:Graph Neural Networks (GNNs) have emerged as a prominent research topic in the field of machine learning. Existing GNN models are commonly categorized into two types: spectral GNNs, which are designed based on polynomial graph filters, and spatial GNNs, which utilize a message-passing scheme as the foundation of the model. For the expressive power and universality of spectral GNNs, a natural approach is to improve the design of basis functions for better approximation ability. As for spatial GNNs, models like Graph Isomorphism Networks (GIN) analyze their expressive power based on Graph Isomorphism Tests. Recently, there have been attempts to establish connections between spatial GNNs and geometric concepts like curvature and cellular sheaves, as well as physical phenomena like oscillators. However, despite the recent progress, there is still a lack of comprehensive analysis regarding the universality of spatial GNNs from the perspectives of geometry and physics. In this paper, we propose MetricGNN (MGNN), a spatial GNN model inspired by the congruent-insensitivity property of classifiers in the classification phase of GNNs. We demonstrate that a GNN model is universal in the spatial domain if it can generate embedding matrices that are congruent to any given embedding matrix. This property is closely related to the Distance Geometry Problem (DGP). Since DGP is an NP-Hard combinatorial optimization problem, we propose optimizing an energy function derived from spring networks and the Multi-Dimensional Scaling (MDS) problem. This approach also allows our model to handle both homophilic and heterophilic graphs. Finally, we propose employing the iteration method to optimize our energy function. We extensively evaluate the effectiveness of our model through experiments conducted on both synthetic and real-world datasets. Our code is available at: <a class="link-external link-https" href="https://github.com/GuanyuCui/MGNN" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the lack of generality of existing graph neural networks (GNNs) in the spatial domain, especially the lack of comprehensive analysis from the geometric and physical perspectives. Specifically, although existing GNN models perform well on certain tasks, their ability to handle homogeneous and heterogeneous graphs is limited, and there is a lack of methods to understand and optimize their generality from the perspective of distance geometry problems (DGP). ### Problem Background 1. **Limitations of Existing GNNs**: - Existing GNN models are mainly divided into two categories: spectral GNNs based on polynomial graph filters and spatial GNNs based on the message - passing mechanism. - Spectral GNNs improve the approximation ability by improving the basis function design, while spatial GNNs such as Graph Isomorphism Networks (GIN) analyze their expressive ability based on graph isomorphism tests. - Although there have been attempts to link spatial GNNs with geometric concepts (such as curvature and cellular layers) and physical phenomena (such as oscillators), a comprehensive analysis of the generality of spatial GNNs from geometric and physical perspectives is still insufficient. 2. **Distance Geometry Problem (DGP)**: - DGP is an NP - Hard combinatorial optimization problem, aiming to determine whether a given distance matrix can be embedded in Euclidean space. - This problem is closely related to the generation of the embedding matrix of GNNs, especially when the embedding matrix needs to maintain congruence with other matrices. ### Main Contributions of the Paper 1. **Proposing the MGNN Model**: - Inspired by the congruence - invariance property in the classification stage of GNNs, the paper proposes a new spatial GNN model - MGNN (MetricGNN), which can generate an embedding matrix that is congruent with any given embedding matrix. - MGNN achieves this goal by minimizing a physically meaningful energy function, which is closely related to the distance geometry problem (DGP) and the multi - dimensional scaling problem (MDS), providing a solid theoretical foundation. 2. **Optimization Method**: - The paper proposes using the fixed - point iteration method to optimize the energy function, thus solving the problem of local minima in non - convex optimization problems. - The experimental results show the effectiveness of the MGNN model on synthetic datasets and real - world datasets. 3. **Handling Homogeneous and Heterogeneous Graphs**: - The MGNN model can handle both homogeneous and heterogeneous graphs simultaneously, which gives it an advantage in a wider range of application scenarios. ### Formula Summary - **Energy Function**: \[ E_p(\mathbf{Z}; \mathbf{M}, E)=\frac{1}{2}\|\mathbf{A} \odot(M(\mathbf{Z})-\mathbf{M})\|_F^2=\sum_{(i, j) \in E} \frac{1}{2}\left(\|\mathbf{Z}_{i:}-\mathbf{Z}_{j:}\|_2^2 - M_{ij}\right)^2 \] - **Final Optimization Target**: \[ L(\mathbf{Z}; \mathbf{Z}^{(0)}, \mathbf{M}, E)=(1 - \alpha)\tilde{E}_p(\mathbf{Z}; \mathbf{M}, E)+\alpha\|\mathbf{Z}-\mathbf{Z}^{(0)}\|_F^2 \] - **Propagation Equation**: \[ \mathbf{Z}^{(k + 1)}=(1 - \alpha)\tilde{\mathbf{A}}\mathbf{Z}^{(k)}+\beta g\mathbf{L}_\mathbf{H}\mathbf{Z}^{(k)}+\alpha\mathbf{Z}^{(0)} \] Through these contributions, the paper not only fills the gap in analyzing the generality of spatial GNNs from geometric and physical perspectives, but also provides new ideas and methods for designing more powerful GNN models.

MGNN: Graph Neural Networks Inspired by Distance Geometry Problem

Geodesic Distance Between Graphs: A Spectral Metric for Assessing the Stability of Graph Neural Networks

Spatial Heterophily Aware Graph Neural Networks

Adaptive curvature exploration geometric graph neural network

Learning the Geodesic Embedding with Graph Neural Networks

Spatio-Spectral Graph Neural Networks

Is Distance Matrix Enough for Geometric Deep Learning?

Revisiting graph neural networks from hybrid regularized graph signal reconstruction

Neighborhood Homophily-based Graph Convolutional Network

Geodesic Graph Neural Network for Efficient Graph Representation Learning

Motif Graph Neural Network

Torsion Graph Neural Networks

Geom-GCN: Geometric Graph Convolutional Networks

HGK-GNN

Joint hyperbolic and Euclidean geometry contrastive graph neural networks

GCNH: A Simple Method For Representation Learning On Heterophilous Graphs

A data-centric framework of improving graph neural networks for knowledge graph embedding

Ha-gnn: a novel graph neural network based on hyperbolic attention

A Survey of Geometric Graph Neural Networks: Data Structures, Models and Applications

Graph Neural Networks with Heterophily

Isometric Graph Neural Networks