Abstract:There has been much recent interest in designing symmetry-aware neural networks (NNs) exhibiting relaxed equivariance. Such NNs aim to interpolate between being exactly equivariant and being fully flexible, affording consistent performance benefits. In a separate line of work, certain structured parameter matrices -- those with displacement structure, characterized by low displacement rank (LDR) -- have been used to design small-footprint NNs. Displacement structure enables fast function and gradient evaluation, but permits accurate approximations via compression primarily to classical convolutional neural networks (CNNs). In this work, we propose a general framework -- based on a novel construction of symmetry-based structured matrices -- to build approximately equivariant NNs with significantly reduced parameter counts. Our framework integrates the two aforementioned lines of work via the use of so-called Group Matrices (GMs), a forgotten precursor to the modern notion of regular representations of finite groups. GMs allow the design of structured matrices -- resembling LDR matrices -- which generalize the linear operations of a classical CNN from cyclic groups to general finite groups and their homogeneous spaces. We show that GMs can be employed to extend all the elementary operations of CNNs to general discrete groups. Further, the theory of structured matrices based on GMs provides a generalization of LDR theory focussed on matrices with cyclic structure, providing a tool for implementing approximate equivariance for discrete groups. We test GM-based architectures on a variety of tasks in the presence of relaxed symmetry. We report that our framework consistently performs competitively compared to approximately equivariant NNs, and other structured matrix-based compression frameworks, sometimes with a one or two orders of magnitude lower parameter count.

Approximately Equivariant Neural Processes

Using and Abusing Equivariance

Approximate Equivariance in Reinforcement Learning

Translation Equivariant Transformer Neural Processes

Symmetry Breaking and Equivariant Neural Networks

What is an equivariant neural network?

Practical Equivariances via Relational Conditional Neural Processes

Equivariant Adaptation of Large Pretrained Models

Does equivariance matter at scale?

Interpreting Equivariant Representations

Relaxed Equivariance via Multitask Learning

Equivariant neural networks for inverse problems

Learning Probabilistic Symmetrization for Architecture Agnostic Equivariance

Equivariant Neural Simulators for Stochastic Spatiotemporal Dynamics

Symmetry-Based Structured Matrices for Efficient Approximately Equivariant Networks

Improved Canonicalization for Model Agnostic Equivariance

Lie Group Decompositions for Equivariant Neural Networks

Graph Neural Networks for Learning Equivariant Representations of Neural Networks

Improving Equivariant Model Training via Constraint Relaxation

Optimal Equivariant Architectures from the Symmetries of Matrix-Element Likelihoods

Breaking the Symmetry: Resolving Symmetry Ambiguities in Equivariant Neural Networks