Abstract:Complex-valued neural networks (CVNNs) have recently shown promising empirical success, for instance for increasing the stability of recurrent neural networks and for improving the performance in tasks with complex-valued inputs, such as in MRI fingerprinting. While the overwhelming success of Deep Learning in the real-valued case is supported by a growing mathematical foundation, such a foundation is still largely lacking in the complex-valued case. We thus analyze the expressivity of CVNNs by studying their approximation properties. Our results yield the first quantitative approximation bounds for CVNNs that apply to a wide class of activation functions including the popular modReLU and complex cardioid activation functions. Precisely, our results apply to any activation function that is smooth but not polyharmonic on some non-empty open set; this is the natural generalization of the class of smooth and non-polynomial activation functions to the complex setting. Our main result shows that the error for the approximation of $C^k$-functions scales as $m^{-k/(2n)}$ for $m \to \infty$ where $m$ is the number of neurons, $k$ the smoothness of the target function and $n$ is the (complex) input dimension. Under a natural continuity assumption, we show that this rate is optimal; we further discuss the optimality when dropping this assumption. Moreover, we prove that the problem of approximating $C^k$-functions using continuous approximation methods unavoidably suffers from the curse of dimensionality.

Approximation by Exponential Type Neural Network Operators

The approximation capabilities of Durrmeyer-type neural network operators

The New Approximation Operators with Sigmoidal Functions

Exponential Sampling Type Kantorovich Max-Product Neural Network Operators

Approximation by Steklov Neural Network Operators

Simultaneous approximation by neural network operators with applications to Voronovskaja formulas

Best Approximation and Inverse Results for Neural Network Operators

Approximation of operators on real line

Approximation of neural network operators based on the Joukowski transformation

Approximation Properties of an Extended Family of the Szász–Mirakjan Beta-Type Operators

Approximation Bounds by Neural Networks in L Ω P [-4Pt]

Approximation Capability of a Novel Neural Network Model for Dynamic Systems

Approximation by bivariate Bernstein–Kantorovich–Stancu operators that reproduce exponential functions

Universal Approximation Abilities of a Modular Differentiable Neural Network

Approximation on Variable Exponent Spaces by Linear Integral Operators

Simultaneous approximation by operators of exponential type

Approximation Rates for Neural Networks with General Activation Functions

Quantitative Approximation for Neural Operators in Nonlinear Parabolic Equations

Approximation by a new kind of -Bernstein–Kantorovich operators

Optimal approximation using complex-valued neural networks

Approximation of discontinuous functions by positive linear operators. A probabilistic approach