Abstract:An ongoing challenge in neural information processing is the following question: how do neurons adjust their connectivity to improve network-level task performance over time (i.e., actualize learning)? It is widely believed that there is a consistent, synaptic-level learning mechanism in specific brain regions, such as the basal ganglia, that actualizes learning. However, the exact nature of this mechanism remains unclear. Here, we investigate the use of universal synaptic-level algorithms in training connectionist models. Specifically, we propose an algorithm based on reinforcement learning (RL) to generate and apply a simple biologically-inspired synaptic-level learning policy for neural networks. In this algorithm, the action space for each synapse in the network consists of a small increase, decrease, or null action on the connection strength. To test our algorithm, we applied it to a multilayer perceptron (MLP) neural network model. This algorithm yields a static synaptic learning policy that enables the simultaneous training of over 20,000 parameters (i.e., synapses) and consistent learning convergence when applied to simulated decision boundary matching and optical character recognition tasks. The trained networks yield character-recognition performance comparable to identically shaped networks trained with gradient descent. The approach has two significant advantages in comparison to traditional gradient-descent-based optimization methods. First, the robustness of our novel method and its lack of reliance on gradient computations opens the door to new techniques for training difficult-to-differentiate artificial neural networks, such as spiking neural networks (SNNs) and recurrent neural networks (RNNs). Second, the method’s simplicity provides a unique opportunity for further development of local information-driven multiagent connectionist models for machine intelligence analogous to cellular automata.

A solution to the learning dilemma for recurrent networks of spiking neurons

Fast learning without synaptic plasticity in spiking neural networks

Beyond spiking networks: The computational advantages of dendritic amplification and input segregation

A Brain-Inspired Spiking Neural Network Model with Temporal Encoding and Learning

Learning arbitrary dynamics in efficient, balanced spiking networks using local plasticity rules

Toward Efficient Processing and Learning with Spikes: New Approaches for Multispike Learning

The brain as an efficient and robust adaptive learner

Using local plasticity rules to train recurrent neural networks

BP-STDP: Approximating backpropagation using spike timing dependent plasticity

Efficient Spike-Driven Learning With Dendritic Event-Based Processing

SpikePropamine: Differentiable Plasticity in Spiking Neural Networks

Temporal Coding in Recurrent Spiking Neural Networks with Synaptic Delay-Weight Plasticity

Gradient-Free Neural Network Training via Synaptic-Level Reinforcement Learning

Gradient Descent for Spiking Neural Networks

Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-Based Optimization to Spiking Neural Networks

A Supervised Learning Rule for Recurrent Spiking Neural Networks with Weighted Spikes.

Smooth Exact Gradient Descent Learning in Spiking Neural Networks

Learning recurrent dynamics in spiking networks

Expanding memory in recurrent spiking networks

Balancing complexity, performance and plausibility to meta learn plasticity rules in recurrent spiking networks.

A Spiking Network for Inference of Relations Trained with Neuromorphic Backpropagation