Abstract:Accumulated detailed knowledge about the neuronal activities in human brains has brought more attention to bio-inspired spiking neural networks (SNNs). In contrast to non-spiking deep neural networks (DNNs), SNNs can encode and transmit spatiotemporal information more efficiently by exploiting biologically realistic and low-power event-driven neuromorphic architectures. However, the supervised learning of SNNs still remains a challenge because the spike-timing-dependent plasticity (STDP) of connected spiking neurons is difficult to implement and interpret in existing backpropagation learning schemes. This paper proposes a fractional-order spike-timing-dependent gradient descent (FO-STDGD) learning model by considering a derived nonlinear activation function that describes the relationship between the quasi-instantaneous firing rate and the temporal membrane potentials of nonleaky integrate-and-fire neurons. The training strategy can be generalized to any fractional orders between 0 and 2 since the FO-STDGD incorporates the fractional gradient descent method into the calculation of spike-timing-dependent loss gradients. The proposed FO-STDGD model is tested on the MNIST and DVS128 Gesture datasets and its accuracy under different network structure and fractional orders is analyzed. It can be found that the classification accuracy increases as the fractional order increases, and specifically, the case of fractional order 1.9 improves by 155% relative to the case of fractional order 1 (traditional gradient descent). In addition, our scheme demonstrates the state-of-the-art computational efficacy for the same SNN structure and training epochs.

A smoothing interval neural networks-based Caputo fractional-order gradient learning algorithm

A Novel Neuro-fuzzy Learning Algorithm for First-Order Takagi–Sugeno Fuzzy Model: Caputo Fractional-Order Gradient Descent Method

Fractional-order gradient descent learning of BP neural networks with Caputo derivative.

Accelerating gradient descent and Adam via fractional gradients

Numerical solution of nonlinear stochastic differential equations with fractional Brownian motion using fractional-order Genocchi deep neural networks

A Novel Gradient Descent Optimizer based on Fractional Order Scheduler and its Application in Deep Neural Networks

Convolutional Neural Networks with Fractional Order Gradient Method.

Study on fast speed fractional order gradient descent method and its application in neural networks

On line parameter estimation based on gradient algorithm for fractional order systems.

Study on Fractional Order Gradient Methods

A Caputo-Type Fractional-Order Gradient Descent Learning of BP Neural Networks.

Exploring the Effects of Caputo Fractional Derivative in Spiking Neural Network Training

An Integer-Fractional Gradient Algorithm for Back Propagation Neural Networks

Convergence Analysis of Caputo-Type Fractional Order Complex-Valued Neural Networks

The Novel Adaptive Fractional Order Gradient Decent Algorithms Design via Robust Control

Fractional-order spike-timing-dependent gradient descent for multi-layer spiking neural networks

An Innovative Fractional Order LMS Based on Variable Initial Value and Gradient Order

Self-Organizing Optimization Based on Caputo’s Fractional Order Gradients

Deep neural network methods for solving forward and inverse problems of time fractional diffusion equations with conformable derivative

A Meshless Approach Based on Fractional Interpolation Theory and Improved Neural Network Bases for Solving Non-Smooth Solution of 2d Fractional Reaction-Diffusion Equation with Distributed Order

A Novel Fractional Order Speedest Gradient Descent Method and its application<sup>*</sup>