Abstract:Motivated by the developing mathematics of deep learning, we build universal functions approximators of continuous maps between arbitrary Polish metric spaces $\mathcal{X}$ and $\mathcal{Y}$ using elementary functions between Euclidean spaces as building blocks. Earlier results assume that the target space $\mathcal{Y}$ is a topological vector space. We overcome this limitation by ``randomization'': our approximators output discrete probability measures over $\mathcal{Y}$. When $\mathcal{X}$ and $\mathcal{Y}$ are Polish without additional structure, we prove very general qualitative guarantees; when they have suitable combinatorial structure, we prove quantitative guarantees for Hölder-like maps, including maps between finite graphs, solution operators to rough differential equations between certain Carnot groups, and continuous non-linear operators between Banach spaces arising in inverse problems. In particular, we show that the required number of Dirac measures is determined by the combinatorial structure of $\mathcal{X}$ and $\mathcal{Y}$. For barycentric $\mathcal{Y}$, including Banach spaces, $\mathbb{R}$-trees, Hadamard manifolds, or Wasserstein spaces on Polish metric spaces, our approximators reduce to $\mathcal{Y}$-valued functions. When the Euclidean approximators are neural networks, our constructions generalize transformer networks, providing a new probabilistic viewpoint of geometric deep learning.

On the probabilistic metrizability of approach spaces

Probabilistic Metric Spaces as enriched categories

A Hofmann-Mislove theorem for approach spaces

On Probalistic Normed Spaces under Τt,l

On the -Spaces of Projective Limits of Probability Measures

On $C$-properties of the space of idempotent robability measures

On Behavioral Metric for Probabilistic Systems: Definition and Approximation Algorithm

Porous Sets for Mutually Nearest Points in Banach Spaces

On Metrics for Probabilistic Systems: Definitions and Algorithms

Sober metric approach spaces

Geometrical properties of the space of idempotent probability measures

Probability Metrics for Tropical Spaces of Different Dimensions

On the Menger Probabilistic Bipolar Metric Spaces: Fixed Point Theorems and Applications

Lipschitz-free spaces over strongly countable-dimensional spaces and approximation properties

On the Borel complexity and the complete metrizability of spaces of metrics

Uncertainty, Imprecise Probabilities and Interval Capacity Measures on a Product Space

Categorical probability spaces, ergodic decompositions, and transitions to equilibrium

Maps of bounded variation from PI spaces to metric spaces

(Metrically) quarter-stratifiable spaces and their applications in the theory of separately continuous functions

On countable tightness type properties of spaces of quasicontinuous functions

An Approximation Theory for Metric Space-Valued Functions With A View Towards Deep Learning