Lie Group Decompositions for Equivariant Neural Networks

Mircea Mironenco,Patrick Forré

2024-07-11

Abstract:Invariance and equivariance to geometrical transformations have proven to be very useful inductive biases when training (convolutional) neural network models, especially in the low-data regime. Much work has focused on the case where the symmetry group employed is compact or abelian, or both. Recent work has explored enlarging the class of transformations used to the case of Lie groups, principally through the use of their Lie algebra, as well as the group exponential and logarithm maps. The applicability of such methods is limited by the fact that depending on the group of interest $G$, the exponential map may not be surjective. Further limitations are encountered when $G$ is neither compact nor abelian. Using the structure and geometry of Lie groups and their homogeneous spaces, we present a framework by which it is possible to work with such groups primarily focusing on the groups $G = \text{GL}^{+}(n, \mathbb{R})$ and $G = \text{SL}(n, \mathbb{R})$, as well as their representation as affine transformations $\mathbb{R}^{n} \rtimes G$. Invariant integration as well as a global parametrization is realized by a decomposition into subgroups and submanifolds which can be handled individually. Under this framework, we show how convolution kernels can be parametrized to build models equivariant with respect to affine transformations. We evaluate the robustness and out-of-distribution generalisation capability of our model on the benchmark affine-invariant classification task, outperforming previous proposals.

Machine Learning

What problem does this paper attempt to address?

This paper explores the use of geometric transformation invariance and equivariance as useful prior knowledge in deep learning, especially when the amount of data is limited. The research focuses on equivariant neural networks on Lie groups, which is an extension of Euclidean groups that deals with a wider range of transformations through Lie algebras, group exponentials, and logarithmic mappings. However, this approach has limitations for certain interested Lie groups, such as non-compact or non-commutative groups, because the group exponential mapping may not be surjective. The paper proposes a framework that effectively handles these types of Lie groups, with a particular focus on the GL+(n,R) and SL(n,R) groups and their representation as affine transformations R^n⋊G. This framework allows for invariant integration and global parameterization, addressing the problem of non-surjectivity of the group exponential mapping, and demonstrates how to construct convolutional kernels equivariant to affine transformations. The paper evaluates the robustness and generalization capability of the model on affine invariant classification tasks, and shows superior performance compared to previous proposals.