Abstract:Complex-valued neural networks (CVNNs) have recently shown promising empirical success, for instance for increasing the stability of recurrent neural networks and for improving the performance in tasks with complex-valued inputs, such as in MRI fingerprinting. While the overwhelming success of Deep Learning in the real-valued case is supported by a growing mathematical foundation, such a foundation is still largely lacking in the complex-valued case. We thus analyze the expressivity of CVNNs by studying their approximation properties. Our results yield the first quantitative approximation bounds for CVNNs that apply to a wide class of activation functions including the popular modReLU and complex cardioid activation functions. Precisely, our results apply to any activation function that is smooth but not polyharmonic on some non-empty open set; this is the natural generalization of the class of smooth and non-polynomial activation functions to the complex setting. Our main result shows that the error for the approximation of $C^k$-functions scales as $m^{-k/(2n)}$ for $m \to \infty$ where $m$ is the number of neurons, $k$ the smoothness of the target function and $n$ is the (complex) input dimension. Under a natural continuity assumption, we show that this rate is optimal; we further discuss the optimality when dropping this assumption. Moreover, we prove that the problem of approximating $C^k$-functions using continuous approximation methods unavoidably suffers from the curse of dimensionality.

Towards an Algebraic Framework For Approximating Functions Using Neural Network Polynomials

Towards a mathematical framework to inform neural network modelling via polynomial regression

Neural Network Approximations of Compositional Functions With Applications to Dynamical Systems

Approximation of RKHS Functionals by Neural Networks

Approximation of Polynomial Functions by Neural Network:Construction of Network and Algorithm of Approximation

Why Deep Neural Networks for Function Approximation?

Fuzzy RBF Neural Network Based Function Approximation

Approximation Bounds by Neural Networks in L Ω P [-4Pt]

An approximation by neural networkswith a fixed weight

Beyond Linearization: On Quadratic and Higher-Order Approximation of Wide Neural Networks

Approximation by Exponential Type Neural Network Operators

Function Approximation with Randomly Initialized Neural Networks for Approximate Model Reference Adaptive Control

Investigation of Neural Networks for Function Approximation

Constructive Trigonometric Function Approximation of Neural Networks

The Estimate for Approximation Error of Neural Networks: A Constructive Approach

Encoder and Decoder Framework for Operator Approximation

Approximations of Continuous Functionals by Neural Networks with Application to Dynamic Systems

Approximation Capability of Interpolation Neural Networks

Optimal approximation using complex-valued neural networks

Chebyshev Polynomial-Based Kolmogorov-Arnold Networks: An Efficient Architecture for Nonlinear Function Approximation

Linearization Turns Neural Operators into Function-Valued Gaussian Processes