Abstract:Leveraging the infinite dimensional neural network architecture we proposed in <a class="link-https" data-arxiv-id="2109.13512v4" href="https://arxiv.org/abs/2109.13512v4">arXiv:2109.13512v4</a> and which can process inputs from Fréchet spaces, and using the universal approximation property shown therein, we now largely extend the scope of this architecture by proving several universal approximation theorems for a vast class of input and output spaces. More precisely, the input space $\mathfrak X$ is allowed to be a general topological space satisfying only a mild condition ("quasi-Polish"), and the output space can be either another quasi-Polish space $\mathfrak Y$ or a topological vector space $E$. Similarly to <a class="link-https" data-arxiv-id="2109.13512v4" href="https://arxiv.org/abs/2109.13512v4">arXiv:2109.13512v4</a>, we show furthermore that our neural network architectures can be projected down to "finite dimensional" subspaces with any desirable accuracy, thus obtaining approximating networks that are easy to implement and allow for fast computation and fitting. The resulting neural network architecture is therefore applicable for prediction tasks based on functional data. To the best of our knowledge, this is the first result which deals with such a wide class of input/output spaces and simultaneously guarantees the numerical feasibility of the ensuing architectures. Finally, we prove an obstruction result which indicates that the category of quasi-Polish spaces is in a certain sense the correct category to work with if one aims at constructing approximating architectures on infinite-dimensional spaces $\mathfrak X$ which, at the same time, have sufficient expressive power to approximate continuous functions on $\mathfrak X$, are specified by a finite number of parameters only and are "stable" with respect to these parameters.

Noncommutative $C^$-algebra Net: Learning Neural Networks with Powerful Product Structure in $C^$-algebra

Quantum Circuit $C^*$-algebra Net

$C^*$-Algebraic Machine Learning: Moving in a New Direction

On $\mathrm{C}^*$-algebras associated to product systems

$C^*$-subproduct and product systems

Neural Hilbert Ladders: Multi-Layer Neural Networks in Function Space

Categorification of Group Equivariant Neural Networks

Stable Tensor Neural Networks for Rapid Deep Learning

Noncompact uniform universal approximation

Neural networks in non-metric spaces

The Dual PC Algorithm and the Role of Gaussianity for Structure Learning of Bayesian Networks

Credit-assignment-based Parallel Ensemble CMAC and Its Applications in Modeling

The Cooperative Network Architecture: Learning Structured Networks as Representation of Sensory Patterns

Collegial Ensembles

Composing Global Optimizers to Reasoning Tasks via Algebraic Objects in Neural Nets

Large subalgebras of crossed products of $C(X)$-algebras

Discrete neural nets and polymorphic learning

Non-linear classification of finite-dimensional simple $C^*$-algebras

Co-universal $C^{\ast}$-algebras for product systems over finite aligned subcategories of groupoids

Compactly-aligned discrete product systems, and generalizations of O_\infty

Noncommutative $C^*$-algebra Net: Learning Neural Networks with Powerful Product Structure in $C^*$-algebra