Abstract:This paper investigates the effectiveness of spiking agents when trained with reinforcement learning (RL) in a challenging multiagent task. In particular, it explores learning through reward-modulated spike-timing dependent plasticity (STDP) and compares it to reinforcement of stochastic synaptic transmission in the general-sum game of the Iterated Prisoner's Dilemma (IPD). More specifically, a computational model is developed where we implement two spiking neural networks as two "selfish" agents learning simultaneously but independently, competing in the IPD game. The purpose of our system (or collective) is to maximise its accumulated reward in the presence of reward-driven competing agents within the collective. This can only be achieved when the agents engage in a behaviour of mutual cooperation during the IPD. Previously, we successfully applied reinforcement of stochastic synaptic transmission to the IPD game. The current study utilises reward-modulated STDP with eligibility trace and results show that the system managed to exhibit the desired behaviour by establishing mutual cooperation between the agents. It is noted that the cooperative outcome was attained after a relatively short learning period which enhanced the accumulation of reward by the system. As in our previous implementation, the successful application of the learning algorithm to the IPD becomes possible only after we extended it with additional global reinforcement signals in order to enhance competition at the neuronal level. Moreover it is also shown that learning is enhanced (as indicated by an increased IPD cooperative outcome) through: (i) strong memory for each agent (regulated by a high eligibility trace time constant) and (ii) firing irregularity produced by equipping the agents' LIF neurons with a partial somatic reset mechanism.

Reinforcement Learning in Spiking Neural Networks with Stochastic and Deterministic Synapses

Spiking neural networks with different reinforcement learning (RL) schemes in a multiagent setting

Robustness of STDP to Spike Timing Jitter.

Toward Efficient Processing and Learning with Spikes: New Approaches for Multispike Learning

Supervised Learning in Spiking Neural Networks with Synaptic Delay-Weight Plasticity

Toward robust and scalable deep spiking reinforcement learning

Incorporating structural plasticity into self-organization recurrent networks for sequence learning

Learning arbitrary dynamics in efficient, balanced spiking networks using local plasticity rules

Exploiting Noise As a Resource for Computation and Learning in Spiking Neural Networks

Coherent noise enables probabilistic sequence replay in spiking neuronal networks

A Neuromorphic Architecture for Reinforcement Learning from Real-Valued Observations

A Spiking Neural Network Structure Implementing Reinforcement Learning

Learning fast changing slow in spiking neural networks

A Simple Probabilistic Spiking Neuron Model with Hebbian Learning Rules.

SpikePropamine: Differentiable Plasticity in Spiking Neural Networks

Improving Robustness of ReRAM-based Spiking Neural Network Accelerator with Stochastic Spike-timing-dependent-plasticity

Computational Modeling of Spiking Neural Network with Learning Rules from STDP and Intrinsic Plasticity

Reinforcement Learning with Feedback-modulated TD-STDP

Deep Reinforcement Learning with Spiking Q-learning

Cooperation of Spike Timing-Dependent and Heterosynaptic Plasticities in Neural Networks: A Fokker-Planck Approach

Spiking network model of the cerebellum as a reinforcement learning machine