Abstract:Complex-valued neural networks (CVNNs) bring in nonholomorphic functions in two ways: (i) through their loss functions and (ii) the widely used activation functions. The derivatives of such functions are defined in Wirtinger calculus. In this paper, we derive two popular algorithms—the gradient descent and the Levenberg-Marquardt (LM) algorithm—for parameter optimization in the feedforward CVNNs using the Wirtinger calculus, which is simpler than the conventional derivation that considers the problem in real domain. While deriving the LM algorithm, we solve and use the result of a least squares problem in the complex domain,$\|\mathbf{b-(Az+Bz^*)}\|_{\underset{\mathbf{z}}{\min}}$, which is more general than the $\|\mathbf{b-Az}\|_{\underset{\mathbf{z}}{\min}}$. Computer simulation results exhibit that as with the real-valued case, the complex-LM algorithm provides much faster learning with higher accuracy than the complex gradient descent algorithm. $|\mathbf{b-(Az+Bz^*)}\|_{\underset{\mathbf{z}}{\min}}$, which is more general than the $\|\mathbf{b-Az}\|_{\underset{\mathbf{z}}{\min}}$. Computer simulation results exhibit that as with the real-valued case, the complex-LM algorithm provides much faster learning with higher accuracy than the complex gradient descent algorithm. $|\mathbf{b-(Az+Bz^*)}\|_{\underset{\mathbf{z}}{\min}}$, which is more general than the $\|\mathbf{b-Az}\|_{\underset{\mathbf{z}}{\min}}$. Computer simulation results exhibit that as with the real-valued case, the complex-LM algorithm provides much faster learning with higher accuracy than the complex gradient descent algorithm. $|\mathbf{b-(Az+Bz^*)}\|_{\underset{\mathbf{z}}{\min}}$, which is more general than the $\|\mathbf{b-Az}\|_{\underset{\mathbf{z}}{\min}}$. Computer simulation results exhibit that as with the real-valued case, the complex-LM algorithm provides much faster learning with higher accuracy than the complex gradient descent algorithm.

Convergence analysis of an augmented algorithm for fully complex-valued neural networks

Convergence Analysis of General Neural Networks under Almost Periodic Stimuli

Convergence analysis of fully complex backpropagation algorithm based on Wirtinger calculus.

Deterministic Convergence of Wirtinger-Gradient Methods for Complex-Valued Neural Networks.

Fully Complex Conjugate Gradient-Based Neural Networks Using Wirtinger Calculus Framework: Deterministic Convergence and Its Application.

Convergence Analysis Of The Augmented Complex Klms Algorithm With Pre-Tuned Dictionary

Convergence Analysis of Three Classes of Split-Complex Gradient Algorithms for Complex-Valued Recurrent Neural Networks

Deterministic Convergence of Complex Mini-Batch Gradient Learning Algorithm for Fully Complex-Valued Neural Networks

An Input Weights Dependent Complex-Valued Learning Algorithm Based on Wirtinger Calculus

Convergence Analysis of Caputo-Type Fractional Order Complex-Valued Neural Networks

Convergence analysis of discrete time recurrent neural networks for linear variational inequality problem

Convergence analysis of wide shallow neural operators within the framework of Neural Tangent Kernel

Convergence analysis of inexact proximal point algorithms on Hadamard manifolds

A Complex Hebbian-type Learning Algorithm with Application to Complex Adaptive ⅡR Filtering

Convergence Analysis of Inverse Iterative Neural Networks with L

Harmonic analysis of neural networks

Convergence Analysis of a Class of Nonsmooth Gradient Systems.

Fully Complex-Valued Wirtinger Conjugate Neural Networks with Generalized Armijo Search

Convergence Analysis of Inverse Iterative Neural Networks with L2 Penalty

Wirtinger calculus based gradient descent and levenberg-marquardt learning algorithms in complex-valued neural networks

Convergence Analysis of the Learning Algorithms for Multi-Valued Neurons