Abstract:Complex-valued neural networks (CVNNs) bring in nonholomorphic functions in two ways: (i) through their loss functions and (ii) the widely used activation functions. The derivatives of such functions are defined in Wirtinger calculus. In this paper, we derive two popular algorithms—the gradient descent and the Levenberg-Marquardt (LM) algorithm—for parameter optimization in the feedforward CVNNs using the Wirtinger calculus, which is simpler than the conventional derivation that considers the problem in real domain. While deriving the LM algorithm, we solve and use the result of a least squares problem in the complex domain,$\|\mathbf{b-(Az+Bz^*)}\|_{\underset{\mathbf{z}}{\min}}$, which is more general than the $\|\mathbf{b-Az}\|_{\underset{\mathbf{z}}{\min}}$. Computer simulation results exhibit that as with the real-valued case, the complex-LM algorithm provides much faster learning with higher accuracy than the complex gradient descent algorithm. $|\mathbf{b-(Az+Bz^*)}\|_{\underset{\mathbf{z}}{\min}}$, which is more general than the $\|\mathbf{b-Az}\|_{\underset{\mathbf{z}}{\min}}$. Computer simulation results exhibit that as with the real-valued case, the complex-LM algorithm provides much faster learning with higher accuracy than the complex gradient descent algorithm. $|\mathbf{b-(Az+Bz^*)}\|_{\underset{\mathbf{z}}{\min}}$, which is more general than the $\|\mathbf{b-Az}\|_{\underset{\mathbf{z}}{\min}}$. Computer simulation results exhibit that as with the real-valued case, the complex-LM algorithm provides much faster learning with higher accuracy than the complex gradient descent algorithm. $|\mathbf{b-(Az+Bz^*)}\|_{\underset{\mathbf{z}}{\min}}$, which is more general than the $\|\mathbf{b-Az}\|_{\underset{\mathbf{z}}{\min}}$. Computer simulation results exhibit that as with the real-valued case, the complex-LM algorithm provides much faster learning with higher accuracy than the complex gradient descent algorithm.

A complex conjugate gradient training algorithm with Barzilai-Borwein stepsize for complex-valued neural networks

A Hybrid Complex Spectral Conjugate Gradient Learning Algorithm for Complex-Valued Data Processing

Adaptive Complex-Valued Stepsize Based Fast Learning of Complex-Valued Neural Networks.

Adaptive Orthogonal Gradient Descent Algorithm for Fully Complex-Valued Neural Networks.

Fully Complex Conjugate Gradient-Based Neural Networks Using Wirtinger Calculus Framework: Deterministic Convergence and Its Application.

Convergence of Batch Split-Complex Backpropagation Algorithm for Complex-Valued Neural Networks

Wirtinger calculus based gradient descent and levenberg-marquardt learning algorithms in complex-valued neural networks

An Efficient Hybrid Incremental Algorithm for Complex-Valued Feedforward Neural Networks

A L-BFGS Based Learning Algorithm for Complex-Valued Feedforward Neural Networks

A Levenberg-Marquardt Algorithm Based Incremental Scheme for Complex-Valued Neural Networks

Deterministic Convergence of Complex Mini-Batch Gradient Learning Algorithm for Fully Complex-Valued Neural Networks

Is a Complex-Valued Stepsize Advantageous in Complex-Valued Gradient Learning Algorithms?

An Efficient Algorithm for Complex-Valued Neural Networks Through Training Input Weights

Fully Complex-Valued Wirtinger Conjugate Neural Networks with Generalized Armijo Search

Convergence Analysis of Three Classes of Split-Complex Gradient Algorithms for Complex-Valued Recurrent Neural Networks

Complex-valued Neural Networks -- Theory and Analysis

Adaptive Stepsize Estimation Based Accelerated Gradient Descent Algorithm for Fully Complex-Valued Neural Networks

An Input Weights Dependent Complex-Valued Learning Algorithm Based on Wirtinger Calculus

The Conjugate Gradient Method with neural network control

A Barzilai-Borwein Conjugate Gradient Method

Deterministic Convergence of Wirtinger-Gradient Methods for Complex-Valued Neural Networks.