Abstract:Artificial neural networks (ANNs), particularly those employing deep learning models, have found widespread application in fields such as computer vision, signal processing, and wireless communications, where complex numbers are crucial. Despite the prevailing use of real-number implementations in current ANN frameworks, there is a growing interest in developing ANNs that utilize complex numbers. This paper presents a comprehensive survey of recent advancements in complex-valued neural networks (CVNNs), focusing on their activation functions (AFs) and learning algorithms. We delve into the extension of the backpropagation algorithm to the complex domain, which enables the training of neural networks with complex-valued inputs, weights, AFs, and outputs. This survey considers three complex backpropagation algorithms: the complex derivative approach, the partial derivatives approach, and algorithms incorporating the Cauchy-Riemann equations. A significant challenge in CVNN design is the identification of suitable nonlinear Complex Valued Activation Functions (CVAFs), due to the conflict between boundedness and differentiability over the entire complex plane as stated by Liouville theorem. We examine both fully complex AFs, which strive for boundedness and differentiability, and split AFs, which offer a practical compromise despite not preserving analyticity. This review provides an in-depth analysis of various CVAFs essential for constructing effective CVNNs. Moreover, this survey not only offers a comprehensive overview of the current state of CVNNs but also contributes to ongoing research and development by introducing a new set of CVAFs (fully complex, split and complex amplitude-phase AFs).

On the Computational Complexities of Complex-valued Neural Networks

Complex-valued Neural Networks -- Theory and Analysis

Comprehensive Survey of Complex-Valued Neural Networks: Insights into Backpropagation and Activation Functions

Optimal approximation using complex-valued neural networks

Complex Network for Complex Problems: A comparative study of CNN and Complex-valued CNN

Computational Complexity Evaluation of Neural Network Applications in Signal Processing

On Complex Valued Convolutional Neural Networks

Deep Complex Networks

Complex-Valued Wavelet Network

Complex-Valued Recurrent Neural Network with IIR Neuron Model: Training and Applications

Evaluation of Complex-Valued Neural Networks on Real-Valued Classification Tasks

Comprehensive Study on Complex-Valued ZNN Models Activated by Novel Nonlinear Functions for Dynamic Complex Linear Equations

Convolutional Neural Networks with Multi-Valued Neurons.

Fully Complex-Valued Dendritic Neuron Model.

A mathematical motivation for complex-valued convolutional networks

A note on the complex and bicomplex valued neural networks

Quantum State Classification Via Complex-Valued Neural Networks

A Hybrid Complex Spectral Conjugate Gradient Learning Algorithm for Complex-Valued Data Processing

Complex-Valued Neural Networks for Data-Driven Signal Processing and Signal Understanding

The Performance Analysis of Complex-Valued Neural Network in Radio Signal Recognition

An optical neural chip for implementing complex-valued neural network