Abstract:Animals rely on different decision strategies when faced with ambiguous or uncertain cues. Depending on the context, decisions may be biased towards events that were most frequently experienced in the past, or be more explorative. A particular type of decision making central to cognition is sequential memory recall in response to ambiguous cues. A previously developed spiking neuronal network implementation of sequence prediction and recall learns complex, high-order sequences in an unsupervised manner by local, biologically inspired plasticity rules. In response to an ambiguous cue, the model deterministically recalls the sequence shown most frequently during training. Here, we present an extension of the model enabling a range of different decision strategies. In this model, explorative behavior is generated by supplying neurons with noise. As the model relies on population encoding, uncorrelated noise averages out, and the recall dynamics remain effectively deterministic. In the presence of locally correlated noise, the averaging effect is avoided without impairing the model performance, and without the need for large noise amplitudes. We investigate two forms of correlated noise occurring in nature: shared synaptic background inputs, and random locking of the stimulus to spatiotemporal oscillations in the network activity. Depending on the noise characteristics, the network adopts various replay strategies. This study thereby provides potential mechanisms explaining how the statistics of learned sequences affect decision making, and how decision strategies can be adjusted after learning.

Maximum diffusion reinforcement learning

Maximum diffusion reinforcement learning

Maximum Entropy Inverse Reinforcement Learning of Diffusion Models with Energy-Based Models

Reward Shaping via Diffusion Process in Reinforcement Learning

Diffusion Spectral Representation for Reinforcement Learning

Coherent noise enables probabilistic sequence replay in spiking neuronal networks

Reinforcement Learning in Spiking Neural Networks with Stochastic and Deterministic Synapses

Maximum Entropy Model-based Reinforcement Learning

Maximum Entropy Diverse Exploration: Disentangling Maximum Entropy Reinforcement Learning

Learning predictive cognitive maps with spiking neurons during behavior and replays

Time-scale invariant contingency yields one-shot reinforcement learning despite extremely long delays to reinforcement

Evolutionary Dispersal of Ecological Species via Multi-Agent Deep Reinforcement Learning

Intrinsic fluctuations of reinforcement learning promote cooperation

Maximum Likelihood Constraint Inference for Inverse Reinforcement Learning

Off-Policy Maximum Entropy RL with Future State and Action Visitation Measures

Continual Diffuser (CoD): Mastering Continual Offline Reinforcement Learning with Experience Rehearsal

Network Diffusions via Neural Mean-Field Dynamics

Large-scale Reinforcement Learning for Diffusion Models

Understanding when Dynamics-Invariant Data Augmentations Benefit Model-Free Reinforcement Learning Updates

Maximum Reward Formulation In Reinforcement Learning

Reinforced diffusions as models of memory-mediated animal movement