Abstract:Abstract Many cognitive processes involve transformations of distributed representations in neural populations, creating a need for population-level models. Recurrent neural network models fulfill this need, but there are many open questions about how their connectivity gives rise to dynamics that solve a task. Here, we present a method for finding the connectivity of networks for which the dynamics are specified to solve a task in an interpretable way. We apply our method to a working memory task by synthesizing a network that implements a drift-diffusion process over a ring-shaped manifold. We also use our method to demonstrate how inputs can be used to control network dynamics for cognitive flexibility and explore the relationship between representation geometry and network capacity. Our work fits within the broader context of understanding neural computations as dynamics over relatively low-dimensional manifolds formed by correlated patterns of neurons. Author Summary Neurons in the brain form intricate networks that can produce a vast array of activity patterns. To support goal-directed behavior, the brain must adjust the connections between neurons so that network dynamics can perform desirable computations on behaviorally relevant variables. A fundamental goal in computational neuroscience is to provide an understanding of how network connectivity aligns the dynamics in the brain to the dynamics needed to track those variables. Here, we develop a mathematical framework for creating recurrent neural network models that can address this problem. Specifically, we derive a set of linear equations that constrain the connectivity to afford a direct mapping of task-relevant dynamics onto network activity. We demonstrate the utility of this technique by creating and analyzing a set of network models that can perform a simple working memory task. We then extend the approach to show how additional constraints can furnish networks whose dynamics are controlled flexibly by external inputs. Finally, we exploit the flexibility of this technique to explore the robustness and capacity limitations of recurrent networks. This network synthesis method provides a powerful means for generating and validating hypotheses about how task-relevant computations can emerge from network dynamics.

Different eigenvalue distributions encode the same temporal tasks in recurrent neural networks

Effect in the spectra of eigenvalues and dynamics of RNNs trained with Excitatory-Inhibitory constraint

Identification of Recurrent Dynamics in Distributed Neural Populations

Ensemble perspective for understanding temporal credit assignment

Engineering recurrent neural networks from task-relevant manifolds and dynamics

Recurrent neural network models for working memory of continuous variables: activity manifolds, connectivity patterns, and dynamic codes

Exploring weight initialization, diversity of solutions, and degradation in recurrent neural networks trained for temporal and decision-making tasks

Short term memory by transient oscillatory dynamics in recurrent neural networks

Learning recurrent dynamics in spiking networks

Learning spatiotemporal signals using a recurrent spiking network that discretizes time

Recurrent network dynamics reconciles visual motion segmentation and integration

Encoding Sensory and Motor Patterns as Time-Invariant Trajectories in Recurrent Neural Networks

A model for cortical activity sequences

Interpreting recurrent neural networks behaviour via excitable network attractors

Investigating the temporal dynamics of electroencephalogram (EEG) microstates using recurrent neural networks

Stimulus-Driven and Spontaneous Dynamics in Excitatory-Inhibitory Recurrent Neural Networks for Sequence Representation

Neural dynamics in a recurrent network model of primary visual cortex

A computational investigation of the relationships between single-neuron and network dynamics in the cerebral cortex

Recurrent cortical networks encode natural sensory statistics via sequence filtering

Recurrence Network Analysis of the Synchronous EEG Time Series in Normal and Epileptic Brains

Synergistic pathways of modulation enable robust task packing within neural dynamics