Abstract:Complex-valued neural networks (CVNNs) bring in nonholomorphic functions in two ways: (i) through their loss functions and (ii) the widely used activation functions. The derivatives of such functions are defined in Wirtinger calculus. In this paper, we derive two popular algorithms—the gradient descent and the Levenberg-Marquardt (LM) algorithm—for parameter optimization in the feedforward CVNNs using the Wirtinger calculus, which is simpler than the conventional derivation that considers the problem in real domain. While deriving the LM algorithm, we solve and use the result of a least squares problem in the complex domain,$\|\mathbf{b-(Az+Bz^*)}\|_{\underset{\mathbf{z}}{\min}}$, which is more general than the $\|\mathbf{b-Az}\|_{\underset{\mathbf{z}}{\min}}$. Computer simulation results exhibit that as with the real-valued case, the complex-LM algorithm provides much faster learning with higher accuracy than the complex gradient descent algorithm. $|\mathbf{b-(Az+Bz^*)}\|_{\underset{\mathbf{z}}{\min}}$, which is more general than the $\|\mathbf{b-Az}\|_{\underset{\mathbf{z}}{\min}}$. Computer simulation results exhibit that as with the real-valued case, the complex-LM algorithm provides much faster learning with higher accuracy than the complex gradient descent algorithm. $|\mathbf{b-(Az+Bz^*)}\|_{\underset{\mathbf{z}}{\min}}$, which is more general than the $\|\mathbf{b-Az}\|_{\underset{\mathbf{z}}{\min}}$. Computer simulation results exhibit that as with the real-valued case, the complex-LM algorithm provides much faster learning with higher accuracy than the complex gradient descent algorithm. $|\mathbf{b-(Az+Bz^*)}\|_{\underset{\mathbf{z}}{\min}}$, which is more general than the $\|\mathbf{b-Az}\|_{\underset{\mathbf{z}}{\min}}$. Computer simulation results exhibit that as with the real-valued case, the complex-LM algorithm provides much faster learning with higher accuracy than the complex gradient descent algorithm.

Adaptive Complex-Valued Stepsize Based Fast Learning of Complex-Valued Neural Networks.

Adaptive Orthogonal Gradient Descent Algorithm for Fully Complex-Valued Neural Networks.

Adaptive Stepsize Estimation Based Accelerated Gradient Descent Algorithm for Fully Complex-Valued Neural Networks

Is a Complex-Valued Stepsize Advantageous in Complex-Valued Gradient Learning Algorithms?

A Levenberg-Marquardt Algorithm Based Incremental Scheme for Complex-Valued Neural Networks

A Hybrid Complex Spectral Conjugate Gradient Learning Algorithm for Complex-Valued Data Processing

An Efficient Hybrid Incremental Algorithm for Complex-Valued Feedforward Neural Networks

A complex conjugate gradient training algorithm with Barzilai-Borwein stepsize for complex-valued neural networks

A L-BFGS Based Learning Algorithm for Complex-Valued Feedforward Neural Networks

Complex-valued Neurons Can Learn More but Slower Than Real-valued Neurons Via Gradient Descent.

Adaptive CL-BFGS Algorithms for Complex-Valued Neural Networks

A Complex Hebbian-type Learning Algorithm with Application to Complex Adaptive ⅡR Filtering

Wirtinger calculus based gradient descent and levenberg-marquardt learning algorithms in complex-valued neural networks

Simple2Complex: Global Optimization by Gradient Descent

Complex-Valued Feedforward Neural Networks Learning Without Backpropagation

Deterministic Convergence of Complex Mini-Batch Gradient Learning Algorithm for Fully Complex-Valued Neural Networks

A Multilayer Complex Neural Network Training Algorithm and Its Application in Adaptive Equalization

An Efficient Algorithm for Complex-Valued Neural Networks Through Training Input Weights

Error Selection Based Training of Fully Complex-Valued Dendritic Neuron Model

An Input Weights Dependent Complex-Valued Learning Algorithm Based on Wirtinger Calculus

A Split-Complex Valued Gradient-Based Descent Neuro-Fuzzy Algorithm for TS System and Its Convergence