Transformer models are gauge invariant: A mathematical connection between AI and particle physics

Leo van Nierop
2024-12-19
Abstract:In particle physics, the fundamental forces are subject to symmetries called gauge invariance. It is a redundancy in the mathematical description of any physical system. In this article I will demonstrate that the transformer architecture exhibits the same properties, and show that the default representation of transformers has partially, but not fully removed the gauge invariance.
Machine Learning,High Energy Physics - Theory
What problem does this paper attempt to address?