Abstract:Symmetry is present throughout nature and continues to play an increasingly central role in physics and machine learning. Fundamental symmetries, such as Poincaré invariance, allow physical laws discovered in laboratories on Earth to be extrapolated to the farthest reaches of the universe. Symmetry is essential to achieving this extrapolatory power in machine learning applications. For example, translation invariance in image classification allows models with fewer parameters, such as convolutional neural networks, to be trained on smaller data sets and achieve state-of-the-art performance. In this paper, we provide a unifying theoretical and methodological framework for incorporating symmetry into machine learning models in three ways: 1. enforcing known symmetry when training a model; 2. discovering unknown symmetries of a given model or data set; and 3. promoting symmetry during training by learning a model that breaks symmetries within a user-specified group of candidates when there is sufficient evidence in the data. We show that these tasks can be cast within a common mathematical framework whose central object is the Lie derivative associated with fiber-linear Lie group actions on vector bundles. We extend and unify several existing results by showing that enforcing and discovering symmetry are linear-algebraic tasks that are dual with respect to the bilinear structure of the Lie derivative. We also propose a novel way to promote symmetry by introducing a class of convex regularization functions based on the Lie derivative and nuclear norm relaxation to penalize symmetry breaking during training of machine learning models. We explain how these ideas can be applied to a wide range of machine learning models including basis function regression, dynamical systems discovery, neural networks, and neural operators acting on fields.

A Commentary on "Breaking Row and Column Symmetries in Matrix Models"

Static Symmetry Breaking in Many-Sorted Finite Model Finding

Finite symmetric systems and their analysis

A Unified Framework to Enforce, Discover, and Promote Symmetry in Machine Learning

Comments on "upper Matrix Bounds for the Discrete Algebraic Riccati Matrix Equation" [with Author's Reply]

Breaking Value Symmetry

Improving Discrete Model Representations via Symmetry Considerations

Numerical Methods for Detecting Symmetries and Commutant Algebras

Exploitation of symmetry in graphs with applications to finite and boundary elements analysis

Discovering Symmetry Breaking in Physical Systems with Relaxed Group Convolution

Linking Rigid Bodies Symmetrically

A group-theoretic formulation for symmetric finite elements

Smoothed Correlators For Symmetric Double-Well Matrix Models: Some Puzzles and Resolutions

Algorithms Transcending the SAT-Symmetry Interface

Rectangular symmetries for coefficients of symmetric functions

satsuma: Structure-based Symmetry Breaking in SAT

Probing the effects of broken symmetries in machine learning

Symmetry Breaking in Quantum Curves and Super Chern-Simons Matrix Models

Symmetries of systems of first order ODEs: Symbolic symmetry computations, mechanistic model construction and applications in biology

Stability analysis of hyper symmetric skeletal structures using group theory

From Physics to Biology by Extending Criticality and Symmetry Breakings