Abstract:The dictionary learning problem can be viewed as a data-driven process to learn a suitable transformation so that data is sparsely represented directly from example data. In this paper, we examine the problem of learning a dictionary that is invariant under a pre-specified group of transformations. Natural settings include Cryo-EM, multi-object tracking, synchronization, pose estimation, etc. We specifically study this problem under the lens of mathematical representation theory. Leveraging the power of non-abelian Fourier analysis for functions over compact groups, we prescribe an algorithmic recipe for learning dictionaries that obey such invariances. We relate the dictionary learning problem in the physical domain, which is naturally modelled as being infinite dimensional, with the associated computational problem, which is necessarily finite dimensional. We establish that the dictionary learning problem can be effectively understood as an optimization instance over certain matrix orbitopes having a particular block-diagonal structure governed by the irreducible representations of the group of symmetries. This perspective enables us to introduce a band-limiting procedure which obtains dimensionality reduction in applications. We provide guarantees for our computational ansatz to provide a desirable dictionary learning outcome. We apply our paradigm to investigate the dictionary learning problem for the groups SO(2) and SO(3). While the SO(2)-orbitope admits an exact spectrahedral description, substantially less is understood about the SO(3)-orbitope. We describe a tractable spectrahedral outer approximation of the SO(3)-orbitope, and contribute an alternating minimization paradigm to perform optimization in this setting. We provide numerical experiments to highlight the efficacy of our approach in learning SO(3)-invariant dictionaries, both on synthetic and on real world data.

Deep Learning Symmetries and Their Lie Groups, Algebras, and Subalgebras from First Principles

Identifying the Group-Theoretic Structure of Machine-Learned Symmetries

Learning Lie Group Symmetry Transformations with Neural Networks

Accelerated Discovery of Machine-Learned Symmetries: Deriving the Exceptional Lie Groups G2, F4 and E6

A Unified Framework to Enforce, Discover, and Promote Symmetry in Machine Learning

Symmetry Discovery for Different Data Types

Learning Infinitesimal Generators of Continuous Symmetries from Data

Latent Space Symmetry Discovery

Generative Adversarial Symmetry Discovery

Lie Algebra Convolutional Neural Networks with Automatic Symmetry Extraction

Finding discrete symmetry groups via Machine Learning

Symmetry Discovery Beyond Affine Transformations

Robust Symmetry Detection via Riemannian Langevin Dynamics

Symmetry Induces Structure and Constraint of Learning

On discrete symmetries of robotics systems: A group-theoretic and data-driven analysis

Symmetry and Orbit Detection Via Lie-Algebra Voting.

Dictionary Learning under Symmetries via Group Representations

Beyond Planar Symmetry: Modeling Human Perception of Reflection and Rotation Symmetries in the Wild

SymmetryLens: A new candidate paradigm for unsupervised symmetry learning via locality and equivariance

Oracle-Preserving Latent Flows

Lie Algebra Canonicalization: Equivariant Neural Operators under arbitrary Lie Groups