Abstract:Topological data analysis (TDA) is a branch of computational mathematics, bridging algebraic topology and data science, that provides compact, noise-robust representations of complex structures. Deep neural networks (DNNs) learn millions of parameters associated with a series of transformations defined by the model architecture, resulting in high-dimensional, difficult-to-interpret internal representations of input data. As DNNs become more ubiquitous across multiple sectors of our society, there is increasing recognition that mathematical methods are needed to aid analysts, researchers, and practitioners in understanding and interpreting how these models' internal representations relate to the final classification. In this paper, we apply cutting edge techniques from TDA with the goal of gaining insight into the interpretability of convolutional neural networks used for image classification. We use two common TDA approaches to explore several methods for modeling hidden-layer activations as high-dimensional point clouds, and provide experimental evidence that these point clouds capture valuable structural information about the model's process. First, we demonstrate that a distance metric based on persistent homology can be used to quantify meaningful differences between layers, and we discuss these distances in the broader context of existing representational similarity metrics for neural network interpretability. Second, we show that a mapper graph can provide semantic insight into how these models organize hierarchical class knowledge at each layer. These observations demonstrate that TDA is a useful tool to help deep learning practitioners unlock the hidden structures of their models.

Some Geometrical and Topological Properties of DNNs' Decision Boundaries

Geometry and Topology of Deep Neural Networks' Decision Boundaries.

A Differential Topological View of Challenges in Learning with Feedforward Neural Networks.

A Geometric Theory for Binary Classification of Finite Datasets by DNNs with Relu Activations

Analysis of Generalizability of Deep Neural Networks Based on the Complexity of Decision Boundary

On the Decision Boundary of Deep Neural Networks.

Understanding the Decision Boundary of Deep Neural Networks: An Empirical Study

Topological Interpretability for Deep-Learning

On Connections Between Regularizations for Improving DNN Robustness

Predicting and Enhancing the Fairness of DNNs with the Curvature of Perceptual Manifolds

Experimental Observations of the Topology of Convolutional Neural Network Activations

How deep learning works --The geometry of deep learning

A Mathematical Principle of Deep Learning: Learn the Geodesic Curve in the Wasserstein Space

Graph Neural Networks and 3-Dimensional Topology

Emergent properties of the local geometry of neural loss landscapes

A mean curvature flow arising in adversarial training

Compositional Curvature Bounds for Deep Neural Networks

Unveiling the Hessian's Connection to the Decision Boundary

Training Dynamics of Deep Network Linear Regions

Geometrization of deep networks for the interpretability of deep learning systems

Empirical Studies on the Properties of Linear Regions in Deep Neural Networks