Abstract:An important problem in machine learning theory is to understand the approximation and generalization properties of two-layer neural networks in high dimensions. To this end, researchers have introduced the Barron space $\mathcal{B}_s(\Omega)$ and the spectral Barron space $\mathcal{F}_s(\Omega)$, where the index $s\in [0,\infty)$ indicates the smoothness of functions within these spaces and $\Omega\subset\mathbb{R}^d$ denotes the input domain. However, the precise relationship between the two types of Barron spaces remains unclear. In this paper, we establish a continuous embedding between them as implied by the following inequality: for any $\delta\in (0,1), s\in \mathbb{N}^{+}$ and $f: \Omega \mapsto\mathbb{R}$, it holds that \[ \delta \|f\|_{\mathcal{F}_{s-\delta}(\Omega)}\lesssim_s \|f\|_{\mathcal{B}_s(\Omega)}\lesssim_s \|f\|_{\mathcal{F}_{s+1}(\Omega)}. \] Importantly, the constants do not depend on the input dimension $d$, suggesting that the embedding is effective in high dimensions. Moreover, we also show that the lower and upper bound are both tight.

Sobolev Embeddings in Infinite Dimensions

Sobolev spaces on locally finite graphs

Quantitative analysis of optimal Sobolev-Lorentz embeddings with $α$-homogeneous weights

An infinite dimensional balanced embedding problem I:existence

Discrete logarithmic Sobolev inequalities in Banach spaces

Poincaré Inequalities in Weighted Sobolev Spaces

Generalized Newton-Leibniz Formula and the Embedding of the Sobolev Functions with Dominating Mixed Smoothness into Hölder Spaces

Spectrality of a class of infinite convolutions with and without compact supports in $\mathbb{R}^d$

Strong Limit Theorems for Extended Independent Random Variables and Extended Negatively Dependent Random Variables under Sub-Linear Expectations

Hyperbolic embedding of infinite-dimensional convex bodies

Embedding derivatives and derivative Area operators of Hardy spaces into Lebesgue spaces

Logarithmic Schrödinger Equations in Infinite Dimensions

Log-Sobolev inequalities and hypercontractivity for Ornstein-Uhlenbeck evolution operators in infinite dimensions

On Sobolev spaces of bounded subanalytic manifolds

A Poincaré Inequality on Loop Spaces

Optimal function spaces and Sobolev embeddings

Embedding Inequalities for Barron-type Spaces

Compact embeddings and Pitt's property for weighted sequence spaces of Sobolev type

Rényi--Sobolev Inequalities and Connections to Spectral Graph Theory

Approximation order of Kolmogorov, Gel'fand, and linear widths for Sobolev embeddings in euclidian measure spaces