Abstract:There has been much recent interest in designing symmetry-aware neural networks (NNs) exhibiting relaxed equivariance. Such NNs aim to interpolate between being exactly equivariant and being fully flexible, affording consistent performance benefits. In a separate line of work, certain structured parameter matrices -- those with displacement structure, characterized by low displacement rank (LDR) -- have been used to design small-footprint NNs. Displacement structure enables fast function and gradient evaluation, but permits accurate approximations via compression primarily to classical convolutional neural networks (CNNs). In this work, we propose a general framework -- based on a novel construction of symmetry-based structured matrices -- to build approximately equivariant NNs with significantly reduced parameter counts. Our framework integrates the two aforementioned lines of work via the use of so-called Group Matrices (GMs), a forgotten precursor to the modern notion of regular representations of finite groups. GMs allow the design of structured matrices -- resembling LDR matrices -- which generalize the linear operations of a classical CNN from cyclic groups to general finite groups and their homogeneous spaces. We show that GMs can be employed to extend all the elementary operations of CNNs to general discrete groups. Further, the theory of structured matrices based on GMs provides a generalization of LDR theory focussed on matrices with cyclic structure, providing a tool for implementing approximate equivariance for discrete groups. We test GM-based architectures on a variety of tasks in the presence of relaxed symmetry. We report that our framework consistently performs competitively compared to approximately equivariant NNs, and other structured matrix-based compression frameworks, sometimes with a one or two orders of magnitude lower parameter count.

An Algorithm for Computing with Brauer's Group Equivariant Neural Network Layers

Brauer's Group Equivariant Neural Networks

A Diagrammatic Approach to Improve Computational Efficiency in Group Equivariant Neural Networks

Categorification of Group Equivariant Neural Networks

Graph Automorphism Group Equivariant Neural Networks

Fast computation of permutation equivariant layers with the partition algebra

Learning Linear Groups in Neural Networks

Lie Group Decompositions for Equivariant Neural Networks

Compact Matrix Quantum Group Equivariant Neural Networks

Grokking Group Multiplication with Cosets

Learning Neural Networks with Two Nonlinear Layers in Polynomial Time

Lie Neurons: Adjoint-Equivariant Neural Networks for Semisimple Lie Algebras

Multivector Neurons: Better and Faster O(n)-Equivariant Clifford Graph Neural Networks

Connecting Permutation Equivariant Neural Networks and Partition Diagrams

Enabling Efficient Equivariant Operations in the Fourier Basis via Gaunt Tensor Products

Finite matrix multiplication algorithms from infinite groups

Geometry of Linear Neural Networks: Equivariance and Invariance under Permutation Groups

Classical and Quantum Algorithms for Orthogonal Neural Networks

Symmetry-Based Structured Matrices for Efficient Approximately Equivariant Networks

Computing equivariant matrices on homogeneous spaces for Geometric Deep Learning and Automorphic Lie Algebras