Abstract:Neural oscillations are ubiquitously observed in many brain areas. One proposed functional role of these oscillations is that they serve as an internal clock, or ‘frame of reference’. Information can be encoded by the timing of neural activity relative to the of such oscillations. In line with this hypothesis, there have been multiple empirical observations of such in the brain. Here we ask: What kind of neural dynamics support phase coding of information with neural oscillations? We tackled this question by analyzing recurrent neural networks (RNNs) that were trained on a working memory task. The networks were given access to an external reference oscillation and tasked to produce an oscillation, such that the phase difference between the reference and output oscillation maintains the identity of transient stimuli. We found that networks converged to stable oscillatory dynamics. Reverse engineering these networks revealed that each phase-coded memory corresponds to a separate limit cycle attractor. We characterized how the stability of the attractor dynamics depends on both reference oscillation amplitude and frequency, properties that can be experimentally observed. To understand the connectivity structures that underlie these dynamics, we showed that trained networks can be described as two phase-coupled oscillators. Using this insight, we condensed our trained networks to a reduced model consisting of two functional modules: One that generates an oscillation and one that implements a coupling function between the internal oscillation and external reference. In summary, by reverse engineering the dynamics and connectivity of trained RNNs, we propose a mechanism by which neural networks can harness reference oscillations for working memory. Specifically, we propose that a phase-coding network generates autonomous oscillations which it couples to an external reference oscillation in a multi-stable fashion.

A robust autoassociative memory with coupled networks of Kuramoto-type oscillators

The functional role of oscillatory dynamics of neocortical circuits: a computational perspective

Artificial Kuramoto Oscillatory Neurons

An Oscillator Neural Network Retrieving Sparsely Coded Phase Patterns

Oscillations in an artificial neural network convert competing inputs into a temporal code

Resonant synchronization and information retrieve from memorized Kuramoto network

Complex dynamics in adaptive phase oscillator networks

Dynamics Analysis and Analog Associative Memory of Networks with LT Neurons.

Coupled Oscillators as a model of Olfactory Network. Importance in Pattern Recognition and Classification tasks

Memorize multiple phase pattern attractors in nonlinear coupled oscillators dynamic via heterogeneous connectivity strength

Cross-Frequency Coupling Increases Memory Capacity in Oscillatory Neural Networks

Dreaming neural networks: Forgetting spurious memories and reinforcing pure ones

Design of oscillatory neural networks by machine learning

Trained recurrent neural networks develop phase-locked limit cycles in a working memory task

Non-linear oscillators with Kuramoto-like local coupling: Complexity analysis and spatiotemporal pattern generation

Recurrent networks recognize patterns with low-dimensional oscillations

Active oscillatory associative memory

Generalized hetero-associative neural networks