Abstract:Symmetry is present throughout nature and continues to play an increasingly central role in physics and machine learning. Fundamental symmetries, such as Poincaré invariance, allow physical laws discovered in laboratories on Earth to be extrapolated to the farthest reaches of the universe. Symmetry is essential to achieving this extrapolatory power in machine learning applications. For example, translation invariance in image classification allows models with fewer parameters, such as convolutional neural networks, to be trained on smaller data sets and achieve state-of-the-art performance. In this paper, we provide a unifying theoretical and methodological framework for incorporating symmetry into machine learning models in three ways: 1. enforcing known symmetry when training a model; 2. discovering unknown symmetries of a given model or data set; and 3. promoting symmetry during training by learning a model that breaks symmetries within a user-specified group of candidates when there is sufficient evidence in the data. We show that these tasks can be cast within a common mathematical framework whose central object is the Lie derivative associated with fiber-linear Lie group actions on vector bundles. We extend and unify several existing results by showing that enforcing and discovering symmetry are linear-algebraic tasks that are dual with respect to the bilinear structure of the Lie derivative. We also propose a novel way to promote symmetry by introducing a class of convex regularization functions based on the Lie derivative and nuclear norm relaxation to penalize symmetry breaking during training of machine learning models. We explain how these ideas can be applied to a wide range of machine learning models including basis function regression, dynamical systems discovery, neural networks, and neural operators acting on fields.

Invariant-based approach to symmetry class detection

An Irreducible Function Basis of Isotropic Invariants of a Third Order Three-Dimensional Symmetric Tensor

Determination of the symmetries of an experimentally determined stiffness tensor; application to acoustic measurements

A Unified Framework to Enforce, Discover, and Promote Symmetry in Machine Learning

Irreducible matrix resolution of the elasticity tensor for symmetry systems

Killing Invariants: An approach to the sub-classification of geometries with symmetry

The distance to cubic symmetry class as a polynomial optimization problem

Symmetry in Signals: A New Insight

Harmonic Functions for Rotational Symmetry Vector Fields

Tensorial harmonic bases of arbitrary order with applications in elasticity, elastoviscoplasticity and texture-based modeling

3D Mixed Invariant and Its Application on Object Classification

Involution symmetry quantification using recurrences

On the Symmetries of 2D Elastic and Hyperelastic Tensors

Planar Symmetry Detection and Quantification using the Extended Persistent Homology Transform

Robust Symmetry Detection via Riemannian Langevin Dynamics

Numerical Methods for Detecting Symmetries and Commutant Algebras

Symmetry properties and explicit solutions of the generalized Weierstrass system

Symmetry’s Applications to Bihamonic Equation and the Generalized Solution of Elasticity Problem

On tensor invariants for integrable cases of Euler, Lagrange and Kovalevskaya rigid body motion

A closed form exact formulation of the spectral representation of a second-order symmetric tensor and of its derivatives

Symmetry Breaking in Symmetric Tensor Decomposition