Abstract:The irreducible complexity of natural phenomena has led Graph Neural Networks to be employed as a standard model to perform representation learning tasks on graph-structured data. While their capacity to capture local and global patterns is remarkable, the implications associated with long-range and higher-order dependencies pose considerable challenges to such models. This work starts with a theoretical framework to reveal the impact of network's width, depth, and graph topology on the over-squashing phenomena in message-passing neural networks. Then, the work drifts towards, higher-order interactions and multi-relational inductive biases via Topological Neural Networks. Such models propagate messages through higher-dimensional structures, providing shortcuts or additional routes for information flow. With this construction, the underlying computational graph is no longer coupled with the input graph structure, thus mitigating the aforementioned bottlenecks while accounting also for higher-order interactions. Inspired by Graph Attention Networks, two topological attention networks are proposed: Simplicial and Cell Attention Networks. The rationale behind these architecture is to leverage the extended notion of neighbourhoods provided by the arrangement of groups of nodes within a simplicial or cell complex to design anisotropic aggregations able to measure the importance of the information coming from different regions of the domain. By doing so, they capture dependencies that conventional Graph Neural Networks might miss. Finally, a multi-way communication scheme is introduced with Enhanced Cellular Isomorphism Networks, which augment topological message passing schemes to enable a direct interactions among groups of nodes arranged in ring-like structures.

Optimising the topology of complex neural networks

Beyond Multilayer Perceptrons: Investigating Complex Topologies in Neural Networks

An Artificial Neural Network Functionalized by Evolution

Finding Better Topologies for Deep Convolutional Neural Networks by Evolution

Random Neural Graph Generation with Structure Evolution.

The Intrinsic Similarity of Topological Structure in Biological Neural Networks.

Characterizing Learning Dynamics of Deep Neural Networks via Complex Networks

Structure and performance of fully connected neural networks: Emerging complex network properties

On Characterizing the Evolution of Embedding Space of Neural Networks using Algebraic Topology

Topology influences performance in the associative memory neural networks

Learning Connectivity of Neural Networks from a Topological Perspective

Multi-scale Topology Optimization using Neural Networks

Brain topology improved spiking neural network for efficient reinforcement learning of continuous control

Learning Optimal Topology for Ad-hoc Robot Networks

Learning to Learn Graph Topologies

Topological Neural Networks: Mitigating the Bottlenecks of Graph Neural Networks via Higher-Order Interactions

Topological exploration of artificial neuronal network dynamics

Estimating Topology of Complex Networks Based on Sparse Bayesian Learning

Random Graph-Based Neuromorphic Learning with a Layer-Weaken Structure

A Complex Network Evolution Model Based on Microscopic Characteristic of Nodes

Evolving Complex Network for Classification Problems