Abstract:The universal approximation theorem is generalised to uniform convergence on the (noncompact) input space $\mathbb{R}^n$. All continuous functions that vanish at infinity can be uniformly approximated by neural networks with one hidden layer, for all activation functions $\varphi$ that are continuous, nonpolynomial, and asymptotically polynomial at $\pm\infty$. When $\varphi$ is moreover bounded, we exactly determine which functions can be uniformly approximated by neural networks, with the following unexpected results. Let $\overline{\mathcal{N}_\varphi^l(\mathbb{R}^n)}$ denote the vector space of functions that are uniformly approximable by neural networks with $l$ hidden layers and $n$ inputs. For all $n$ and all $l\geq2$, $\overline{\mathcal{N}_\varphi^l(\mathbb{R}^n)}$ turns out to be an algebra under the pointwise product. If the left limit of $\varphi$ differs from its right limit (for instance, when $\varphi$ is sigmoidal) the algebra $\overline{\mathcal{N}_\varphi^l(\mathbb{R}^n)}$ ($l\geq2$) is independent of $\varphi$ and $l$, and equals the closed span of products of sigmoids composed with one-dimensional projections. If the left limit of $\varphi$ equals its right limit, $\overline{\mathcal{N}_\varphi^l(\mathbb{R}^n)}$ ($l\geq1$) equals the (real part of the) commutative resolvent algebra, a C*-algebra which is used in mathematical approaches to quantum theory. In the latter case, the algebra is independent of $l\geq1$, whereas in the former case $\overline{\mathcal{N}_\varphi^2(\mathbb{R}^n)}$ is strictly bigger than $\overline{\mathcal{N}_\varphi^1(\mathbb{R}^n)}$.

Universal Approximation Theorem and error bounds for quantum neural networks and quantum reservoirs

Non-asymptotic Approximation Error Bounds of Parameterized Quantum Circuits

Quantum Neural Networks As Universal Function Approximators: Theory and Implementation

On the Universal Approximability and Complexity Bounds of Deep Learning in Hybrid Quantum-Classical Computing

A duplication-free quantum neural network for universal approximation

A quantum neural network framework for scalable quantum circuit approximation of unitary matrices

Universal approximation of continuous functions with minimal quantum circuits

Universal Approximation Property of Quantum Machine Learning Models in Quantum-Enhanced Feature Spaces

Quantum consistent neural/tensor networks for photonic circuits with strongly/weakly entangled states

Approximating Ground States by Neural Network Quantum States

Approximating Ground States by Neural Network Quantum States.

Noncompact uniform universal approximation

Neural network representation of quantum systems

Neural-network states for the classical simulation of quantum computing

QuApprox: A Framework for Benchmarking the Approximability of Variational Quantum Circuit

Universal Approximation Theorem for Neural Networks

Universal Approximation Theorem for Vector- and Hypercomplex-Valued Neural Networks

Universality conditions of unified classical and quantum reservoir computing

Quantum Artificial Neural Networks with Applications

SuperEncoder: Towards Universal Neural Approximate Quantum State Preparation

Quantum Fourier Transformation Using Quantum Reservoir Computing Network