Abstract:Researchers are actively trying to gain better insights into the representational properties of convolutional neural networks for guiding better network designs and for interpreting a network's computational nature. Gaining such insights can be an arduous task due to the number of parameters in a network and the complexity of a network's architecture. Current approaches of neural network interpretation include Bayesian probabilistic interpretations and information theoretic interpretations. In this study, we take a different approach to studying convolutional neural networks by proposing an abstract algebraic interpretation using finite transformation semigroup theory. Specifically, convolutional layers are broken up and mapped to a finite space. The state space of the proposed finite transformation semigroup is then defined as a single element within the convolutional layer, with the acting elements defined by surrounding state elements combined with convolution kernel elements. Generators of the finite transformation semigroup are defined to complete the interpretation. We leverage this approach to analyze the basic properties of the resulting finite transformation semigroup to gain insights on the representational properties of convolutional neural networks, including insights into quantized network representation. Such a finite transformation semigroup interpretation can also enable better understanding outside of the confines of fixed lattice data structures, thus useful for handling data that lie on irregular lattices. Furthermore, the proposed abstract algebraic interpretation is shown to be viable for interpreting convolutional operations within a variety of convolutional neural network architectures.

A Generalization of the Convolution Theorem and its Connections to Non-Stationarity and the Graph Frequency Domain

Fading memory and the convolution theorem

A Generalized Convolution Theorem for the Special Affine Fourier Transform and Its Application to Filtering

Graphon Filters: Graph Signal Processing in the Limit

Generalized convolution theorem associated with fractional Fourier transform

The faces of Convolution: from the Fourier theory to algebraic signal processing

Transferability of Spectral Graph Convolutional Neural Networks

Understanding convolution on graphs via energies

Seeing Convolution Through the Eyes of Finite Transformation Semigroup Theory: An Abstract Algebraic Interpretation of Convolutional Neural Networks

Generalized Convolution Operator Associated with the (k, a)-Generalized Fourier Transform on the Real Line and Applications

Group Action Equivariance And Generalized Convolution In Multi-Layer Neural Networks

On the Transferability of Spectral Graph Filters

Convolution calculus on white noise spaces and Feynman graph representation of generalized renormalization flows

Advancing Graph Convolutional Networks via General Spectral Wavelets

A Convolution and Correlation Theorem for the Linear Canonical Transform and Its Application

Generalized Graphon Process: Convergence of Graph Frequencies in Stretched Cut Distance

Modified Stieltjes Transform and Generalized Convolutions

Generalized Convolution Properties Based on the Modified Mittag-Leffler Function

HoloNets: Spectral Convolutions do extend to Directed Graphs

Lie Group Algebra Convolutional Filters

Graph-Time Convolutional Neural Networks