Universal Approximation of Operators with Transformers and Neural Integral Operators

Emanuele Zappala,Maryam Bagherian
2024-09-02
Abstract:We study the universal approximation properties of transformers and neural integral operators for operators in Banach spaces. In particular, we show that the transformer architecture is a universal approximator of integral operators between Hölder spaces. Moreover, we show that a generalized version of neural integral operators, based on the Gavurin integral, are universal approximators of arbitrary operators between Banach spaces. Lastly, we show that a modified version of transformer, which uses Leray-Schauder mappings, is a universal approximator of operators between arbitrary Banach spaces.
Machine Learning,Numerical Analysis
What problem does this paper attempt to address?