What problem does this paper attempt to address?

The problem this paper attempts to address is the approximation properties of single hidden layer neural networks when weights and thresholds are constrained. Specifically, the authors investigate whether such neural networks can densely approximate any function in the space of continuous functions when weights vary in only a finite number of directions and thresholds come from an open interval. Additionally, the authors demonstrate the density results for neural networks with specifically constructed activation functions and a fixed number of neurons. ### Background of the Paper Over the past 30 years, artificial neural networks have been a hot research area. Neural networks have been widely applied in various fields such as computer science, finance, medicine, engineering, and physics. Among them, the single hidden layer perceptron model has attracted much attention due to its powerful ability to approximate arbitrary functions. This paper mainly explores the density (or arbitrary precision approximation) problem of single hidden layer neural networks under constrained weights and thresholds. ### Main Contributions 1. **Necessary and Sufficient Conditions**: The authors obtained measure-theoretic conditions under which single hidden layer neural networks, with weights varying in only a finite number of directions and thresholds from an open interval, can densely approximate any function in the space of continuous functions. 2. **Smooth Activation Functions**: The authors proved the existence of smooth activation functions that allow neural networks to maintain density even with a fixed number of neurons. ### Research Methods - **Measure-Theoretic Methods**: By introducing the concept of measure theory, particularly measures orthogonal to ridge functions, the authors established the necessary and sufficient conditions for the density of neural networks. - **Geometric Methods**: Using geometric concepts such as lightning bolts, the authors further explored the density of neural networks under specific conditions. ### Conclusion - **Density Conditions**: The condition for single hidden layer neural networks, with weights varying in only a finite number of directions and thresholds from an open interval, to densely approximate any function in the space of continuous functions is the absence of closed paths or closed lightning bolts. - **Smooth Activation Functions**: There exist smooth activation functions that allow neural networks to maintain density even with a fixed number of neurons. ### Application Prospects These results not only theoretically determine the effective boundaries of neural networks under constrained weights and thresholds but also provide a theoretical basis for further optimizing neural network structures. For example, in practical applications, suitable activation functions and network structures can be selected based on these theoretical results to improve the approximation performance and computational efficiency of neural networks.

Measure theoretic results for approximation by neural networks with limited weights

Approximation error of single hidden layer neural networks with fixed weights

Universal Approximation Theorem for Neural Networks

Weighted Sobolev Approximation Rates for Neural Networks on Unbounded Domains

Numerical Approximation Capacity of Neural Networks with Bounded Parameters: Do Limits Exist, and How Can They Be Measured?

On the approximation by single hidden layer feedforward neural networks with fixed weights

Convergence and Approximation of Invariant Measures for Neural Field Lattice Models under Noise Perturbation

An approximation by neural networkswith a fixed weight

Lp Simultaneous Approximation by Neural Networks with One Hidden Layer

An elementary proof of a universal approximation theorem

Characterization of Degree of Approximation for Neural Networks with One Hidden Layer

Universal approximation results for neural networks with non-polynomial activation function over non-compact domains

Sharp Bounds on the Approximation Rates, Metric Entropy, and $n$-widths of Shallow Neural Networks

Approximations of Continuous Functionals by Neural Networks with Application to Dynamic Systems

Size and depth of monotone neural networks: interpolation and approximation

Function approximation by neural nets in the mean-field regime: Entropic regularization and controlled McKean-Vlasov dynamics

Neural networks in non-metric spaces

New advances in universal approximation with neural networks of minimal width

Deep Network Approximation: Achieving Arbitrary Accuracy with Fixed Number of Neurons

Multi-layer random features and the approximation power of neural networks