Abstract:We derive universal approximation results for the class of (countably) $m$-rectifiable measures. Specifically, we prove that $m$-rectifiable measures can be approximated as push-forwards of the one-dimensional Lebesgue measure on $[0,1]$ using ReLU neural networks with arbitrarily small approximation error in terms of Wasserstein distance. What is more, the weights in the networks under consideration are quantized and bounded and the number of ReLU neural networks required to achieve an approximation error of $\varepsilon$ is no larger than $2^{b(\varepsilon)}$ with $b(\varepsilon)=\mathcal{O}(\varepsilon^{-m}\log^2(\varepsilon))$. This result improves Lemma IX.4 in Perekrestenko et al. as it shows that the rate at which $b(\varepsilon)$ tends to infinity as $\varepsilon$ tends to zero equals the rectifiability parameter $m$, which can be much smaller than the ambient dimension. We extend this result to countably $m$-rectifiable measures and show that this rate still equals the rectifiability parameter $m$ provided that, among other technical assumptions, the measure decays exponentially on the individual components of the countably $m$-rectifiable support set.

What problem does this paper attempt to address?

The core problem that this paper attempts to solve is the general approximation problem regarding $m$-rectifiable measures. Specifically, the authors prove how to approximate $m$-rectifiable measures by push - forwards of the one - dimensional Lebesgue measure on the interval $[0,1]$ through ReLU neural networks, and can achieve an arbitrarily small approximation error under the Wasserstein distance. In addition, they also show that the weights of these neural networks are quantized and bounded, and the number of required ReLU neural networks has a relationship with the approximation error $\varepsilon$ as $2^{b(\varepsilon)}$, where $b(\varepsilon)=O(\varepsilon^{-m}\log_2(\varepsilon))$. ### Main Contributions 1. **Improving Existing Results**: Compared with previous work (such as Perekrestenko et al.), the results in this paper show that as $\varepsilon$ approaches zero, the growth rate of $b(\varepsilon)$ is equal to the rectifiable parameter $m$, rather than the ambient dimension $n$. This leads to a better approximation effect in data measures supported on low - dimensional objects. 2. **Extension to Countable $m$-Rectifiable Measures**: The article further generalizes this result to countable $m$-rectifiable measures and proves that under certain technical assumptions, $b(\varepsilon)$ still satisfies the above relationship. 3. **Explicit Construction of Neural Networks**: The authors not only provide theoretical approximation results but also describe in detail how to explicitly construct ReLU neural networks for approximation, including the quantization of their structures and weights. ### Application Background Generative models are widely used for the approximate representation of high - dimensional complex data, especially in cases where it is difficult to obtain a large amount of training data. They expand the data set by generating synthetic data. Deep generative models use deep neural networks to generate these model measures, and the research results in this paper provide a new theoretical basis and practical method for such models. ### Formula Summary - **Relationship between Approximation Error and Network Quantity**: \[ b(\varepsilon)=O(\varepsilon^{-m}\log_2(\varepsilon)) \] - **Application of the Triangle Inequality of Wasserstein Distance**: \[ W_1(\nu, (\Phi\circ\Sigma)_{\#}\lambda)\leq W_1(\nu, \Phi_{\#}\mu)+\text{Lip}(\Phi)W_1(\mu, \Sigma_{\#}\lambda) \] ### Conclusion Approximating $m$-rectifiable measures through ReLU neural networks is not only of great theoretical significance but also provides an effective tool for generative models in practical applications. Especially for data measures supported on low - dimensional objects, this method can significantly reduce the required computational resources and storage space.

Generating Rectifiable Measures through Neural Networks

Rates of Approximation by ReLU Shallow Neural Networks

New advances in universal approximation with neural networks of minimal width

Approximation in $L^p(μ)$ with deep ReLU neural networks

Uniform Approximation with Quadratic Neural Networks

Deep Neural Networks: Multi-Classification and Universal Approximation

Optimal Approximation Rates for Deep ReLU Neural Networks on Sobolev and Besov Spaces

On the growth of the parameters of approximating ReLU neural networks

Measure theoretic results for approximation by neural networks with limited weights

Minimum Width of Leaky-ReLU Neural Networks for Uniform Universal Approximation

Learning Distributions Generated by One-Layer ReLU Networks

Minimum width for universal approximation using ReLU networks on compact domain

Deep Network Approximation: Achieving Arbitrary Accuracy with Fixed Number of Neurons

An elementary proof of a universal approximation theorem

A general approximation lower bound in $L^p$ norm, with applications to feed-forward neural networks

Noncompact uniform universal approximation

Approximation of functions from Korobov spaces by shallow neural networks

Three Quantization Regimes for ReLU Networks

Numerical Approximation Capacity of Neural Networks with Bounded Parameters: Do Limits Exist, and How Can They Be Measured?

Implicit Hypersurface Approximation Capacity in Deep ReLU Networks

Bridging the Gap Between Approximation and Learning via Optimal Approximation by ReLU MLPs of Maximal Regularity