Abstract:Complex-valued neural networks (CVNNs) bring in nonholomorphic functions in two ways: (i) through their loss functions and (ii) the widely used activation functions. The derivatives of such functions are defined in Wirtinger calculus. In this paper, we derive two popular algorithms—the gradient descent and the Levenberg-Marquardt (LM) algorithm—for parameter optimization in the feedforward CVNNs using the Wirtinger calculus, which is simpler than the conventional derivation that considers the problem in real domain. While deriving the LM algorithm, we solve and use the result of a least squares problem in the complex domain,$\|\mathbf{b-(Az+Bz^*)}\|_{\underset{\mathbf{z}}{\min}}$, which is more general than the $\|\mathbf{b-Az}\|_{\underset{\mathbf{z}}{\min}}$. Computer simulation results exhibit that as with the real-valued case, the complex-LM algorithm provides much faster learning with higher accuracy than the complex gradient descent algorithm. $|\mathbf{b-(Az+Bz^*)}\|_{\underset{\mathbf{z}}{\min}}$, which is more general than the $\|\mathbf{b-Az}\|_{\underset{\mathbf{z}}{\min}}$. Computer simulation results exhibit that as with the real-valued case, the complex-LM algorithm provides much faster learning with higher accuracy than the complex gradient descent algorithm. $|\mathbf{b-(Az+Bz^*)}\|_{\underset{\mathbf{z}}{\min}}$, which is more general than the $\|\mathbf{b-Az}\|_{\underset{\mathbf{z}}{\min}}$. Computer simulation results exhibit that as with the real-valued case, the complex-LM algorithm provides much faster learning with higher accuracy than the complex gradient descent algorithm. $|\mathbf{b-(Az+Bz^*)}\|_{\underset{\mathbf{z}}{\min}}$, which is more general than the $\|\mathbf{b-Az}\|_{\underset{\mathbf{z}}{\min}}$. Computer simulation results exhibit that as with the real-valued case, the complex-LM algorithm provides much faster learning with higher accuracy than the complex gradient descent algorithm.

Convergence Analysis of the Learning Algorithms for Multi-Valued Neurons

Finite convergence of the learning algorithms for a modified multi-valued neuron

Convergence Analysis of General Neural Networks under Almost Periodic Stimuli

Convergence Analysis of Two-layer Neural Networks with ReLU Activation

Convergence Analysis of Three Classes of Split-Complex Gradient Algorithms for Complex-Valued Recurrent Neural Networks

Convergence Theorems for the Kohonen Feature Mapping Algorithms with VLRPs

Convergence analysis of discrete time recurrent neural networks for linear variational inequality problem

Convergence of Gradient Method for A Fully Recurrent Neural Network

Convergence Analysis of Online Gradient Method for BP Neural Networks

A Convergence Analysis of Gradient Descent for Deep Linear Neural Networks

Analysis of Boundedness and Convergence of Online Gradient Method for Two-Layer Feedforward Neural Networks

Convergence of Gradient Algorithm for BP Neural Network with Multiple Output Units

Globally Convergent Neural Networks

A Class of Gradient Algorithms with Variable Learning Rates and Convergence Analysis for Feedforward Neural Networks Training

Convergence Theorems for a Class of Learning Algorithms with VLRPs

Convergence Analysis of Multilayer Feedforward Networks Trained with Penalty Terms: A review

Convergence Analysis of an Improved Extreme Learning Machine Based on Gradient Descent Method

Convergence of Stochastic Gradient Descent in Deep Neural Network

Convergence analysis of wide shallow neural operators within the framework of Neural Tangent Kernel

Deterministic Convergence of Complex Mini-Batch Gradient Learning Algorithm for Fully Complex-Valued Neural Networks

Wirtinger calculus based gradient descent and levenberg-marquardt learning algorithms in complex-valued neural networks