Abstract:Accumulated detailed knowledge about the neuronal activities in human brains has brought more attention to bio-inspired spiking neural networks (SNNs). In contrast to non-spiking deep neural networks (DNNs), SNNs can encode and transmit spatiotemporal information more efficiently by exploiting biologically realistic and low-power event-driven neuromorphic architectures. However, the supervised learning of SNNs still remains a challenge because the spike-timing-dependent plasticity (STDP) of connected spiking neurons is difficult to implement and interpret in existing backpropagation learning schemes. This paper proposes a fractional-order spike-timing-dependent gradient descent (FO-STDGD) learning model by considering a derived nonlinear activation function that describes the relationship between the quasi-instantaneous firing rate and the temporal membrane potentials of nonleaky integrate-and-fire neurons. The training strategy can be generalized to any fractional orders between 0 and 2 since the FO-STDGD incorporates the fractional gradient descent method into the calculation of spike-timing-dependent loss gradients. The proposed FO-STDGD model is tested on the MNIST and DVS128 Gesture datasets and its accuracy under different network structure and fractional orders is analyzed. It can be found that the classification accuracy increases as the fractional order increases, and specifically, the case of fractional order 1.9 improves by 155% relative to the case of fractional order 1 (traditional gradient descent). In addition, our scheme demonstrates the state-of-the-art computational efficacy for the same SNN structure and training epochs.

Study on fast speed fractional order gradient descent method and its application in neural networks

A Novel Fractional Order Speedest Gradient Descent Method and its application<sup>*</sup>

Study on Two-Stage Fractional Order Gradient Descend Method

Design of Generalized Fractional Order Gradient Descent Method

Generalization of the Gradient Method with Fractional Order Gradient Direction

Convolutional Neural Networks with Fractional Order Gradient Method.

Study on Fractional Order Gradient Methods

A Novel Gradient Descent Optimizer based on Fractional Order Scheduler and its Application in Deep Neural Networks

Applications of fractional calculus in learned optimization

The Improved Stochastic Fractional Order Gradient Descent Algorithm

An Integer-Fractional Gradient Algorithm for Back Propagation Neural Networks

Fractional-order stochastic gradient descent method with momentum and energy for deep neural networks

The Novel Adaptive Fractional Order Gradient Decent Algorithms Design via Robust Control

Quadratic Gradient: Combining Gradient Algorithms and Newton's Method as One

A smoothing interval neural networks-based Caputo fractional-order gradient learning algorithm

Fractional Order Gradient Methods for a General Class of Convex Functions

On The Unified Design Of Accelerated Gradient Descent

A novel perspective to gradient method: the fractional order approach

Exploring the Effects of Caputo Fractional Derivative in Spiking Neural Network Training

Accelerated Gradient-free Neural Network Training by Multi-convex Alternating Optimization

Fractional-order spike-timing-dependent gradient descent for multi-layer spiking neural networks