Abstract:Complex-valued neural network is a kind of learning model which can deal with problems in complex domain. Fully complex extreme learning machine (CELM) is a much faster training algorithm than the complex backpropagation (CBP) scheme. However, it is at the cost of using more hidden nodes to obtain the comparable performance. An upper-layer-solution-aware algorithm has been proposed for training single-hidden layer feedforward neural networks, which performs much better than its counterparts, pseudo-inverse learning (PIL)/extreme learning machine and gradient decent-based backpropagation neural networks. Consequently, there exist two challenges that need to be dealt with: 1) How to combine the advantages of CBP and CELM to develop a novel complex learning algorithm? and 2) What is the convergent behavior of the presented algorithm? In this article, an input weights dependent complex-valued (IWDCV) learning algorithm based on Wirtinger calculus has been proposed, which effectively solves the nonanalytic problem of the common activation functions during training neural networks. In addition, the monotonicity of the error function and the deterministic convergence of the proposed model have been strictly proved, which theoretically guarantee the efficiency and effectiveness of the given model, IWDCV. Finally, for real and complex-valued problems, a variety of simulations have been done to demonstrate the comparable performance of the proposed algorithm which support the theoretical observations as well.

Neural Networks with Complex-Valued Weights Have No Spurious Local Minima

Spurious Local Minima Are Common for Deep Neural Networks with Piecewise Linear Activations

Complex Critical Points of Deep Linear Neural Networks

Avoiding Spurious Local Minima in Deep Quadratic Networks

Exact Solutions of a Deep Linear Network

On the Omnipresence of Spurious Local Minima in Certain Neural Network Training Problems

Depth Creates No Bad Local Minima

Spurious Local Minima of Deep ReLU Neural Networks in the Neural Tangent Kernel Regime

Complex-valued Neurons Can Learn More but Slower Than Real-valued Neurons Via Gradient Descent.

Evaluation of Complex-Valued Neural Networks on Real-Valued Classification Tasks

Pure and Spurious Critical Points: a Geometric Study of Linear Networks

Stuck in a What? Adventures in Weight Space

On the Complexity of Learning Neural Networks

Piecewise linear activations substantially shape the loss surfaces of neural networks

Asymmetric Valleys: Beyond Sharp and Flat Local Minima.

Invariance of Weight Distributions in Rectified MLPs

Optimal approximation using complex-valued neural networks

Towards Understanding Theoretical Advantages of Complex-Reaction Networks

Asymptotical Stability for a Class of Complex-Valued Projective Neural Network.

An Input Weights Dependent Complex-Valued Learning Algorithm Based on Wirtinger Calculus

Global stability of complex-valued neural networks with both leakage time delay and discrete time delay on time scales