Abstract:The irreducible complexity of natural phenomena has led Graph Neural Networks to be employed as a standard model to perform representation learning tasks on graph-structured data. While their capacity to capture local and global patterns is remarkable, the implications associated with long-range and higher-order dependencies pose considerable challenges to such models. This work starts with a theoretical framework to reveal the impact of network's width, depth, and graph topology on the over-squashing phenomena in message-passing neural networks. Then, the work drifts towards, higher-order interactions and multi-relational inductive biases via Topological Neural Networks. Such models propagate messages through higher-dimensional structures, providing shortcuts or additional routes for information flow. With this construction, the underlying computational graph is no longer coupled with the input graph structure, thus mitigating the aforementioned bottlenecks while accounting also for higher-order interactions. Inspired by Graph Attention Networks, two topological attention networks are proposed: Simplicial and Cell Attention Networks. The rationale behind these architecture is to leverage the extended notion of neighbourhoods provided by the arrangement of groups of nodes within a simplicial or cell complex to design anisotropic aggregations able to measure the importance of the information coming from different regions of the domain. By doing so, they capture dependencies that conventional Graph Neural Networks might miss. Finally, a multi-way communication scheme is introduced with Enhanced Cellular Isomorphism Networks, which augment topological message passing schemes to enable a direct interactions among groups of nodes arranged in ring-like structures.

On the Topological Properties of the Certain Neural Networks.

Computing topological indices of probabilistic neural network

Topological Properties of Four-Layered Neural Networks.

M-Polynomial Method For Topological Indices Of 3-Layered Probabilistic Neural Networks

Study of Topological Behavior of Some Computer Related Graphs

The Intrinsic Similarity of Topological Structure in Biological Neural Networks.

Topological data analysis of the firings of a network of stochastic spiking neurons

On topological indices of certain interconnection networks

A Comprehensive Review of Deep Neural Network Interpretation Using Topological Data Analysis

Learning Connectivity of Neural Networks from a Topological Perspective

Topological Data Analysis of Neural Network Layer Representations

Topological Probability and Connection Strength Induced Activity in Complex Neural Networks

Topological Similarity Between Artificial and Biological Neural Networks.

On Topological Properties of Third type of Hex Derived Networks

On Topological Properties of Planar Octahedron Networks

The Grey Complex Networks and Design of Some Topological Indexes of Grey Indirect Networks

Conditional Uncertainty Quantification for Tensorized Topological Neural Networks

Exposition and Interpretation of the Topology of Neural Networks

On topological indices of certain families of graphs

Topological Neural Networks: Mitigating the Bottlenecks of Graph Neural Networks via Higher-Order Interactions

Topological Neural Networks go Persistent, Equivariant, and Continuous