On the Completeness of Invariant Geometric Deep Learning Models

Zian Li,Xiyuan Wang,Shijia Kang,Muhan Zhang

2024-07-08

Abstract:Invariant models, one important class of geometric deep learning models, are capable of generating meaningful geometric representations by leveraging informative geometric features in point clouds. These models are characterized by their simplicity, good experimental results and computational efficiency. However, their theoretical expressive power still remains unclear, restricting a deeper understanding of the potential of such models. In this work, we concentrate on characterizing the theoretical expressiveness of a wide range of invariant models. We first rigorously bound the expressiveness of the most classic invariant model, message-passing neural networks incorporating distance (DisGNN), restricting its unidentifiable cases to be only highly symmetric point clouds. We then show that GeoNGNN, the geometric counterpart of one of the simplest subgraph graph neural networks (subgraph GNNs), can effectively break these corner cases' symmetry and thus achieve E(3)-completeness. By leveraging GeoNGNN as a theoretical tool, we further prove that: 1) most subgraph GNNs developed in traditional graph learning can be seamlessly extended to geometric scenarios with E(3)-completeness; 2) DimeNet, GemNet and SphereNet, three well-established invariant models, are also all capable of achieving E(3)-completeness. Our theoretical results fill the gap in the theoretical power of invariant models, contributing to a rigorous and comprehensive understanding of their capabilities. We also empirically evaluated GeoNGNN, the simplest model within the large E(3)-complete family we established, which achieves competitive results to models relying on high-order invariant/equivariant representations on molecule-relevant tasks.

Machine Learning,Artificial Intelligence

What problem does this paper attempt to address?

The paper discusses the completeness problem of geometric deep learning models, especially the capacity of invariant models. Invariant models can utilize geometric features in point clouds to generate meaningful geometric representations, but their theoretical expressive power is still unclear. The research focuses on characterizing the theoretical expressive power of a series of invariant models, especially the expressive boundary of Message Passing Neural Networks (DisGNN). The paper proves that DisGNN may have recognition problems in highly symmetrical point clouds and proposes GeoNGNN, which is a geometric version of Subgraph GNN that can break these symmetries and achieve E(3)-completeness. Furthermore, the authors demonstrate that several other established invariant models, such as DimeNet, GemNet, and SphereNet, can also achieve E(3)-completeness. These theoretical results fill the gap in the theoretical capabilities of invariant models and contribute to a deeper understanding of their potentials. In addition, experiments show that GeoNGNN competes competitively with models that rely on higher-order invariant/covariant representations in molecular-related tasks. The work of the paper emphasizes that the bottleneck of invariant models in geometric settings may lie in generalization rather than expressive power, expanding the design space of geometric deep learning models.