Abstract:Topological deep learning (TDL) facilitates learning from data represented by topological structures. The primary model utilized in this setting is higher-order message-passing (HOMP), which extends traditional graph message-passing neural networks (MPNN) to diverse topological domains. Given the significant expressivity limitations of MPNNs, our paper aims to explore both the strengths and weaknesses of HOMP's expressive power and subsequently design novel architectures to address these limitations. We approach this from several perspectives: First, we demonstrate HOMP's inability to distinguish between topological objects based on fundamental topological and metric properties such as diameter, orientability, planarity, and homology. Second, we show HOMP's limitations in fully leveraging the topological structure of objects constructed using common lifting and pooling operators on graphs. Finally, we compare HOMP's expressive power to hypergraph networks, which are the most extensively studied TDL methods. We then develop two new classes of TDL models: multi-cellular networks (MCN) and scalable multi-cellular networks (SMCN). These models draw inspiration from expressive graph architectures. While MCN can reach full expressivity but is highly unscalable, SMCN offers a more scalable alternative that still mitigates many of HOMP's expressivity limitations. Finally, we construct a synthetic dataset, where TDL models are tasked with separating pairs of topological objects based on basic topological properties. We demonstrate that while HOMP is unable to distinguish between any of the pairs in the dataset, SMCN successfully distinguishes all pairs, empirically validating our theoretical findings. Our work opens a new design space and new opportunities for TDL, paving the way for more expressive and versatile models.

Topological Blind Spots: Understanding and Extending Topological Deep Learning Through the Lens of Expressivity

Architectures of Topological Deep Learning: A Survey of Message-Passing Topological Neural Networks

Position: Topological Deep Learning is the New Frontier for Relational Learning

Topological deep learning: a review of an emerging paradigm

A Comprehensive Review of Deep Neural Network Interpretation Using Topological Data Analysis

Experimental Observations of the Topology of Convolutional Neural Network Activations

Topological Neural Networks: Mitigating the Bottlenecks of Graph Neural Networks via Higher-Order Interactions

E(n) Equivariant Topological Neural Networks

Attending to Topological Spaces: The Cellular Transformer

Higher-Order Topological Directionality and Directed Simplicial Neural Networks

Learning Connectivity of Neural Networks from a Topological Perspective

A Transformative Topological Representation for Link Modeling, Prediction and Cross-Domain Network Analysis

Topological Interpretability for Deep-Learning

Topological Signal Processing and Learning: Recent Advances and Future Challenges

TopoTune : A Framework for Generalized Combinatorial Complex Neural Networks

TopoBenchmarkX: A Framework for Benchmarking Topological Deep Learning

Topological Learning in Multi-Class Data Sets

Topological Methods in Machine Learning: A Tutorial for Practitioners

Topological learning in multiclass data sets

Topological Neural Networks go Persistent, Equivariant, and Continuous

Topological Learning for Brain Networks