Abstract:Topological deep learning (TDL) facilitates learning from data represented by topological structures. The primary model utilized in this setting is higher-order message-passing (HOMP), which extends traditional graph message-passing neural networks (MPNN) to diverse topological domains. Given the significant expressivity limitations of MPNNs, our paper aims to explore both the strengths and weaknesses of HOMP's expressive power and subsequently design novel architectures to address these limitations. We approach this from several perspectives: First, we demonstrate HOMP's inability to distinguish between topological objects based on fundamental topological and metric properties such as diameter, orientability, planarity, and homology. Second, we show HOMP's limitations in fully leveraging the topological structure of objects constructed using common lifting and pooling operators on graphs. Finally, we compare HOMP's expressive power to hypergraph networks, which are the most extensively studied TDL methods. We then develop two new classes of TDL models: multi-cellular networks (MCN) and scalable multi-cellular networks (SMCN). These models draw inspiration from expressive graph architectures. While MCN can reach full expressivity but is highly unscalable, SMCN offers a more scalable alternative that still mitigates many of HOMP's expressivity limitations. Finally, we construct a synthetic dataset, where TDL models are tasked with separating pairs of topological objects based on basic topological properties. We demonstrate that while HOMP is unable to distinguish between any of the pairs in the dataset, SMCN successfully distinguishes all pairs, empirically validating our theoretical findings. Our work opens a new design space and new opportunities for TDL, paving the way for more expressive and versatile models.

Topological deep learning: a review of an emerging paradigm

Topology Preserving Dictionary Learning for Pattern Classification.

Position: Topological Deep Learning is the New Frontier for Relational Learning

A Comprehensive Review of Deep Neural Network Interpretation Using Topological Data Analysis

Architectures of Topological Deep Learning: A Survey of Message-Passing Topological Neural Networks

Topological Deep Learning: A New Dimension in Gastroenterology for Metabolic Dysfunction-Associated Fatty Liver

Topological data analysis and machine learning

Topological Methods in Machine Learning: A Tutorial for Practitioners

Topological Blind Spots: Understanding and Extending Topological Deep Learning Through the Lens of Expressivity

Topological data analysis in medical imaging: current state of the art

Understanding Deep Neural Networks Using Topological Data Analysis

Computational Topology for Data Analysis

Topological Data Analysis with Applications

An Introduction to Topological Data Analysis: Fundamental and Practical Aspects for Data Scientists

Topological Data Analysis (TDA) for Time Series

TopoBenchmarkX: A Framework for Benchmarking Topological Deep Learning

TopOC: Topological Deep Learning for Ovarian and Breast Cancer Diagnosis

Experimental Observations of the Topology of Convolutional Neural Network Activations

Topological Signal Processing and Learning: Recent Advances and Future Challenges

Attending to Topological Spaces: The Cellular Transformer

Using Topological Data Analysis to Process Time-series Data: A Persistent Homology Way