Abstract:The computational properties of neural systems are often thought to be implemented in terms of their network dynamics. Hence, recovering the system dynamics from experimentally observed neuronal time series, like multiple single-unit (MSU) recordings or neuroimaging data, is an important step toward understanding its computations. Ideally, one would not only seek a state space representation of the dynamics, but would wish to have access to its governing equations for in-depth analysis. Recurrent neural networks (RNNs) are a computationally powerful and dynamically universal formal framework which has been extensively studied from both the computational and the dynamical systems perspective. Here we develop a semi-analytical maximum-likelihood estimation scheme for piecewise-linear RNNs (PLRNNs) within the statistical framework of state space models, which accounts for noise in both the underlying latent dynamics and the observation process. The Expectation-Maximization algorithm is used to infer the latent state distribution, through a global Laplace approximation, and the PLRNN parameters iteratively. After validating the procedure on toy examples, the approach is applied to MSU recordings from the rodent anterior cingulate cortex obtained during performance of a classical working memory task, delayed alternation. A model with 5 states turned out to be sufficient to capture the essential computational dynamics underlying task performance, including stimulus-selective delay activity. The estimated models were rarely multi-stable, but rather were tuned to exhibit slow dynamics in the vicinity of a bifurcation point. In summary, the present work advances a semi-analytical (thus reasonably fast) maximum-likelihood estimation framework for PLRNNs that may enable to recover the relevant dynamics underlying observed neuronal time series, and directly link them to computational properties.

Reverse engineering recurrent neural networks with Jacobian switching linear dynamical systems

Recurrent switching linear dynamical systems

Parsing neural dynamics with infinite recurrent switching linear dynamical systems

Structure-Exploiting variational inference for recurrent switching linear dynamical systems

Decomposed Linear Dynamical Systems (dLDS) for learning the latent components of neural dynamics

A State Space Approach for Piecewise-Linear Recurrent Neural Networks for Reconstructing Nonlinear Dynamics from Neural Measurements

Identifying nonlinear dynamical systems with multiple time scales and long-range dependencies

Modeling Latent Neural Dynamics with Gaussian Process Switching Linear Dynamical Systems

Almost-Linear RNNs Yield Highly Interpretable Symbolic Codes in Dynamical Systems Reconstruction

Linear RNNs Provably Learn Linear Dynamical Systems

Recurrent Switching Dynamical Systems Models for Multiple Interacting Neural Populations

Identifying nonlinear dynamical systems via generative recurrent neural networks with applications to fMRI

Reverse engineering recurrent networks for sentiment classification reveals line attractor dynamics

Where is all the nonlinearity: flexible nonlinear modeling of behaviorally relevant neural dynamics using recurrent neural networks

Probing the Relationship Between Latent Linear Dynamical Systems and Low-Rank Recurrent Neural Network Models

Automatic Structure and Parameter Training Methods for Modeling of Mechanical System by Recurrent Neural Networks

Identification of Recurrent Dynamics in Distributed Neural Populations

Towards Scalable and Stable Parallelization of Nonlinear RNNs

Non-normal Recurrent Neural Network (nnRNN): learning long time dependencies while improving expressivity with transient dynamics