Abstract:Complex-valued neural networks (CVNNs) bring in nonholomorphic functions in two ways: (i) through their loss functions and (ii) the widely used activation functions. The derivatives of such functions are defined in Wirtinger calculus. In this paper, we derive two popular algorithms—the gradient descent and the Levenberg-Marquardt (LM) algorithm—for parameter optimization in the feedforward CVNNs using the Wirtinger calculus, which is simpler than the conventional derivation that considers the problem in real domain. While deriving the LM algorithm, we solve and use the result of a least squares problem in the complex domain,$\|\mathbf{b-(Az+Bz^*)}\|_{\underset{\mathbf{z}}{\min}}$, which is more general than the $\|\mathbf{b-Az}\|_{\underset{\mathbf{z}}{\min}}$. Computer simulation results exhibit that as with the real-valued case, the complex-LM algorithm provides much faster learning with higher accuracy than the complex gradient descent algorithm. $|\mathbf{b-(Az+Bz^*)}\|_{\underset{\mathbf{z}}{\min}}$, which is more general than the $\|\mathbf{b-Az}\|_{\underset{\mathbf{z}}{\min}}$. Computer simulation results exhibit that as with the real-valued case, the complex-LM algorithm provides much faster learning with higher accuracy than the complex gradient descent algorithm. $|\mathbf{b-(Az+Bz^*)}\|_{\underset{\mathbf{z}}{\min}}$, which is more general than the $\|\mathbf{b-Az}\|_{\underset{\mathbf{z}}{\min}}$. Computer simulation results exhibit that as with the real-valued case, the complex-LM algorithm provides much faster learning with higher accuracy than the complex gradient descent algorithm. $|\mathbf{b-(Az+Bz^*)}\|_{\underset{\mathbf{z}}{\min}}$, which is more general than the $\|\mathbf{b-Az}\|_{\underset{\mathbf{z}}{\min}}$. Computer simulation results exhibit that as with the real-valued case, the complex-LM algorithm provides much faster learning with higher accuracy than the complex gradient descent algorithm.

Wirtinger calculus based gradient descent and levenberg-marquardt learning algorithms in complex-valued neural networks

An Input Weights Dependent Complex-Valued Learning Algorithm Based on Wirtinger Calculus

A L-BFGS Based Learning Algorithm for Complex-Valued Feedforward Neural Networks

Complex-valued Neurons Can Learn More but Slower Than Real-valued Neurons Via Gradient Descent.

Complex-valued Neural Networks -- Theory and Analysis

A Complex Hebbian-type Learning Algorithm with Application to Complex Adaptive ⅡR Filtering

Comprehensive Survey of Complex-Valued Neural Networks: Insights into Backpropagation and Activation Functions

A Split-Complex Valued Gradient-Based Descent Neuro-Fuzzy Algorithm for TS System and Its Convergence

Complex-Valued Filtering Based on the Minimization of Complex-Error Entropy.

A Multilayer Complex Neural Network Training Algorithm and Its Application in Adaptive Equalization

Two Bicomplex and One Multicomplex Least Mean Square algorithms

Complex-Valued Recurrent Neural Network with IIR Neuron Model: Training and Applications

Simple2Complex: Global Optimization by Gradient Descent

Is a Complex-Valued Stepsize Advantageous in Complex-Valued Gradient Learning Algorithms?

Optimal approximation using complex-valued neural networks

A Mean-Field Analysis of Neural Stochastic Gradient Descent-Ascent for Functional Minimiax Optimization

Adaptive Levenberg–Marquardt Algorithm: A New Optimization Strategy for Levenberg–Marquardt Neural Networks

A Mean-Field Analysis of Neural Stochastic Gradient Descent-Ascent for Functional Minimax Optimization

ML-descent: An optimization algorithm for full-waveform inversion using machine learning

Why Learning of Large-Scale Neural Networks Behaves Like Convex Optimization

Concaving Space Algorithm For Neural Network