Abstract:The irreducible complexity of natural phenomena has led Graph Neural Networks to be employed as a standard model to perform representation learning tasks on graph-structured data. While their capacity to capture local and global patterns is remarkable, the implications associated with long-range and higher-order dependencies pose considerable challenges to such models. This work starts with a theoretical framework to reveal the impact of network's width, depth, and graph topology on the over-squashing phenomena in message-passing neural networks. Then, the work drifts towards, higher-order interactions and multi-relational inductive biases via Topological Neural Networks. Such models propagate messages through higher-dimensional structures, providing shortcuts or additional routes for information flow. With this construction, the underlying computational graph is no longer coupled with the input graph structure, thus mitigating the aforementioned bottlenecks while accounting also for higher-order interactions. Inspired by Graph Attention Networks, two topological attention networks are proposed: Simplicial and Cell Attention Networks. The rationale behind these architecture is to leverage the extended notion of neighbourhoods provided by the arrangement of groups of nodes within a simplicial or cell complex to design anisotropic aggregations able to measure the importance of the information coming from different regions of the domain. By doing so, they capture dependencies that conventional Graph Neural Networks might miss. Finally, a multi-way communication scheme is introduced with Enhanced Cellular Isomorphism Networks, which augment topological message passing schemes to enable a direct interactions among groups of nodes arranged in ring-like structures.

Algebraic Topological Networks via the Persistent Local Homology Sheaf

A functorial Dowker theorem and persistent homology of asymmetric networks

Cosheaf Theoretical Constructions in Networks and Persistent Homology

Topological Neural Networks: Mitigating the Bottlenecks of Graph Neural Networks via Higher-Order Interactions

The importance of the whole: Topological data analysis for the network neuroscientist

Sheaf Hypergraph Networks

Topological Data Analysis of Neural Network Layer Representations

Learning Persistent Homology of 3D Point Clouds

Topo-MLP : A Simplicial Network Without Message Passing

Persistent hyperdigraph homology and persistent hyperdigraph Laplacians

On Characterizing the Evolution of Embedding Space of Neural Networks using Algebraic Topology

Neural Sheaf Diffusion: A Topological Perspective on Heterophily and Oversmoothing in GNNs

Topology Applied to Machine Learning: From Global to Local

Persistent Topological Laplacians -- a Survey

Functional Loops: Monitoring Functional Organization of Deep Neural Networks Using Algebraic Topology

Representing the Topology of Complex Networks Based on Graph Embedding

On localizing topological algebras

The Topology of Local Computing in Networks

Topological Learning for Brain Networks

Topological Neural Networks go Persistent, Equivariant, and Continuous