Abstract:Neural oscillations are ubiquitously observed in many brain areas. One proposed functional role of these oscillations is that they serve as an internal clock, or ‘frame of reference’. Information can be encoded by the timing of neural activity relative to the of such oscillations. In line with this hypothesis, there have been multiple empirical observations of such in the brain. Here we ask: What kind of neural dynamics support phase coding of information with neural oscillations? We tackled this question by analyzing recurrent neural networks (RNNs) that were trained on a working memory task. The networks were given access to an external reference oscillation and tasked to produce an oscillation, such that the phase difference between the reference and output oscillation maintains the identity of transient stimuli. We found that networks converged to stable oscillatory dynamics. Reverse engineering these networks revealed that each phase-coded memory corresponds to a separate limit cycle attractor. We characterized how the stability of the attractor dynamics depends on both reference oscillation amplitude and frequency, properties that can be experimentally observed. To understand the connectivity structures that underlie these dynamics, we showed that trained networks can be described as two phase-coupled oscillators. Using this insight, we condensed our trained networks to a reduced model consisting of two functional modules: One that generates an oscillation and one that implements a coupling function between the internal oscillation and external reference. In summary, by reverse engineering the dynamics and connectivity of trained RNNs, we propose a mechanism by which neural networks can harness reference oscillations for working memory. Specifically, we propose that a phase-coding network generates autonomous oscillations which it couples to an external reference oscillation in a multi-stable fashion.

Recurrent networks recognize patterns with low-dimensional oscillations

Trained recurrent neural networks develop phase-locked limit cycles in a working memory task

Stimulus-Driven and Spontaneous Dynamics in Excitatory-Inhibitory Recurrent Neural Networks for Sequence Representation

A Recurrent Neural Network for Rhythmic Timing

Recurrent Neural Network Modeling and Low-Dimensional Analysis of Parkinson's Dynamics

The functional role of oscillatory dynamics of neocortical circuits: a computational perspective

Recurrent Spectral Network (RSN): shaping the basin of attraction of a discrete map to reach automated classification

Complex Recurrent Spectral Network

A robust autoassociative memory with coupled networks of Kuramoto-type oscillators

Short term memory by transient oscillatory dynamics in recurrent neural networks

Pattern Recognition Computation in a Spiking Neural Network with Temporal Encoding and Learning

Oscillations in an artificial neural network convert competing inputs into a temporal code

Recurrent neural networks as neuro-computational models of human speech recognition

Characterising representation dynamics in recurrent neural networks for object recognition

Learning-induced Pattern Classification in a Chaotic Neural Network

Theoretical Study of Oscillator Neurons in Recurrent Neural Networks

Interpreting recurrent neural networks behaviour via excitable network attractors

Understanding Recurrent Neural Networks Using Nonequilibrium Response Theory

Recurrence Resonance -- Noise-Enhanced Dynamics in Recurrent Neural Networks

Dynamical mechanisms of how an RNN keeps a beat, uncovered with a low-dimensional reduced model