Abstract:Even as machine learning exceeds human-level performance on many applications, the generality, robustness, and rapidity of the brain's learning capabilities remain unmatched. How cognition arises from neural activity is a central open question in neuroscience, inextricable from the study of intelligence itself. A simple formal model of neural activity was proposed in Papadimitriou [2020] and has been subsequently shown, through both mathematical proofs and simulations, to be capable of implementing certain simple cognitive operations via the creation and manipulation of assemblies of neurons. However, many intelligent behaviors rely on the ability to recognize, store, and manipulate temporal sequences of stimuli (planning, language, navigation, to list a few). Here we show that, in the same model, time can be captured naturally as precedence through synaptic weights and plasticity, and, as a result, a range of computations on sequences of assemblies can be carried out. In particular, repeated presentation of a sequence of stimuli leads to the memorization of the sequence through corresponding neural assemblies: upon future presentation of any stimulus in the sequence, the corresponding assembly and its subsequent ones will be activated, one after the other, until the end of the sequence. Finally, we show that any finite state machine can be learned in a similar way, through the presentation of appropriate patterns of sequences. Through an extension of this mechanism, the model can be shown to be capable of universal computation. We support our analysis with a number of experiments to probe the limits of learning in this model in key ways. Taken together, these results provide a concrete hypothesis for the basis of the brain's remarkable abilities to compute and learn, with sequences playing a vital role.

A Neural Model of Rule Discovery with Relatively Short-Term Sequence Memory

Sequential Memory: a Putative Neural and Synaptic Dynamical Mechanism.

A hippocampus CA3 spiking neural network model for storage and retrieval of sequential memory

Recurrent Network Models Of Sequence Generation And Memory

Predictive Attractor Models

Retrieval of branching sequences in associative memory model with common external input and bias input

Forecasting Future Action Sequences with Neural Memory Networks

Sequence memory for prediction, inference and behaviour

Neuro-Symbolic Temporal Point Processes

Relational recurrent neural networks

A Neural Dynamic Model based on Activation Diffusion and a Micro-Explanation for Cognitive Operations

Dynamic emergence of relational structure network in human brains

Temporal-Sequential Learning with Columnar-Structured Spiking Neural Networks.

The neural signature of information regularity in temporally extended event sequences

Long Sequence Hopfield Memory

Sequence Structure Organizes Items in Varied Latent States of Working Memory Neural Network.

Roles of Dynamic Linkage of Stable Attractors Across Cortical Networks in Recalling Long-Term Memory

Spontaneous Temporal Grouping Neural Network for Long-Term Memory Modeling

Computation with Sequences in a Model of the Brain

Sequence Classification with Neural Conditional Random Fields

Long sequence Hopfield memory*