Abstract:Recently, the equivariance of models with respect to a group action has become an important topic of research in machine learning. Analysis of the built-in equivariance of existing neural network architectures, as well as the study of building models that explicitly "bake in" equivariance, have become significant research areas in their own right. However, imbuing an architecture with a specific group equivariance imposes a strong prior on the types of data transformations that the model expects to see. While strictly-equivariant models enforce symmetries, real-world data does not always conform to such strict equivariances. In such cases, the prior of strict equivariance can actually prove too strong and cause models to underperform. Therefore, in this work we study a closely related topic, that of almost equivariance. We provide a definition of almost equivariance and give a practical method for encoding almost equivariance in models by appealing to the Lie algebra of a Lie group. Specifically, we define Lie algebra convolutions and demonstrate that they offer several benefits over Lie group convolutions, including being well-defined for non-compact Lie groups having non-surjective exponential map. From there, we demonstrate connections between the notions of equivariance and isometry and those of almost equivariance and almost isometry. We prove two existence theorems, one showing the existence of almost isometries within bounded distance of isometries of a manifold, and another showing the converse for Hilbert spaces. We extend these theorems to prove the existence of almost equivariant manifold embeddings within bounded distance of fully equivariant embedding functions, subject to certain constraints on the group action and the function class. Finally, we demonstrate the validity of our approach by benchmarking against datasets in fully equivariant and almost equivariant settings.

Solutions of equivariance for iterative differential equations

Regularity, synthesis, rigidity and analytic classification for linear ordinary differential equations of second order

Group Invariant Solutions Without Transversality

Invariance of second order ordinary differential equations under two-dimensional affine subalgebras of EP Lie algebra

Infinitely Many Solutions for Diffusion Equations Without Symmetry

Linear determining equations, differential constraints and invariant solutions

Equivariant Filters are Equivariant

On the Infinite -Order Solutions of Second Order Linear Differential Equations

Group Invariant Solutions in Mathematical Physics and Differential Geometry

LIE GROUPS AND NUMERICAL SOLUTIONS OF DIFFERENTIAL EQUATIONS: INVARIANT DISCRETIZATION VERSUS DIFFERENTIAL APPROXIMATION

On invariants of second-order ordinary differential equations $y''=f(x,y,y')$ via point transformations

Invariants associated with linear ordinary differential equations

Existence of Three Solutions for Nonlinear Operator Equations and Applications to Second-Order Differential Equations

Equivalence for Differential Equations

Patterns of Non-Stationary Solutions to Symmetric Systems of Second-Order Differential Equations

Lie remarkable partial differential equations characterized by Lie algebras of point symmetries

Some Remarks on the Solution of Linearisable Second-Order Ordinary Differential Equations via Point Transformations

Entire Solutions Of Certain Non-Linear Differential-Difference Equations

Admissible transformations and Lie symmetries of linear systems of second-order ordinary differential equations

Oscillation Result for Some Non-homogeneous Linear Differential Equations and Its Application

Almost Equivariance via Lie Algebra Convolutions