Abstract:Sequence learning, prediction and replay have been proposed to constitute the universal computations performed by the neocortex. The Hierarchical Temporal Memory (HTM) algorithm realizes these forms of computation. It learns sequences in an unsupervised and continuous manner using local learning rules, permits a context specific prediction of future sequence elements, and generates mismatch signals in case the predictions are not met. While the HTM algorithm accounts for a number of biological features such as topographic receptive fields, nonlinear dendritic processing, and sparse connectivity, it is based on abstract discrete-time neuron and synapse dynamics, as well as on plasticity mechanisms that can only partly be related to known biological mechanisms. Here, we devise a continuous-time implementation of the temporal-memory (TM) component of the HTM algorithm, which is based on a recurrent network of spiking neurons with biophysically interpretable variables and parameters. The model learns high-order sequences by means of a structural Hebbian synaptic plasticity mechanism supplemented with a rate-based homeostatic control. In combination with nonlinear dendritic input integration and local inhibitory feedback, this type of plasticity leads to the dynamic self-organization of narrow sequence-specific subnetworks. These subnetworks provide the substrate for a faithful propagation of sparse, synchronous activity, and, thereby, for a robust, context specific prediction of future sequence elements as well as for the autonomous replay of previously learned sequences. By strengthening the link to biology, our implementation facilitates the evaluation of the TM hypothesis based on electrophysiological and behavioral data.

Temporal-Sequential Learning with Columnar-Structured Spiking Neural Networks.

Spatio-Temporal Sequential Memory Model with Mini-Column Neural Network

A hippocampus CA3 spiking neural network model for storage and retrieval of sequential memory

A Hierarchical Organized Memory Model Using Spiking Neurons

Minicolumn-Based Episodic Memory Model With Spiking Neurons, Dendrites and Delays

A Brain-Inspired Spiking Neural Network Model with Temporal Encoding and Learning

Unleashing the Potential of Spiking Neural Networks for Sequential Modeling with Contextual Embedding.

A Spiking Neural Network Model For Associative Memory Using Temporal Codes

Spontaneous Temporal Grouping Neural Network for Long-Term Memory Modeling

A Spiking Neural Network System for Robust Sequence Recognition

How the Brain Formulates Memory: A Spatio-Temporal Model Research Frontier.

Rapid Feedforward Computation by Temporal Encoding and Learning with Spiking Neurons

Sequential Memory: a Putative Neural and Synaptic Dynamical Mechanism.

Sequence learning, prediction, and replay in networks of spiking neurons

Biologically Plausible Sequence Learning with Spiking Neural Networks.

A Hierarchically Organized Memory Model With Temporal Population Coding

Temporal dendritic heterogeneity incorporated with spiking neural networks for learning multi-timescale dynamics

A New Hierarchical Temporal Memory Based on Recurrent Learning Unit

Pattern Recognition Computation in a Spiking Neural Network with Temporal Encoding and Learning

Sequential Memory with Temporal Predictive Coding

Sequence memory for prediction, inference and behaviour