Abstract:Data-driven inference of the generative dynamics underlying a set of observed time series is of growing interest in machine learning and the natural sciences. In neuroscience, such methods promise to alleviate the need to handcraft models based on biophysical principles and allow to automatize the inference of inter-individual differences in brain dynamics. Recent breakthroughs in training techniques for state space models (SSMs) specifically geared toward dynamical systems (DS) reconstruction (DSR) enable to recover the underlying system including its geometrical (attractor) and long-term statistical invariants from even short time series. These techniques are based on control-theoretic ideas, like modern variants of teacher forcing (TF), to ensure stable loss gradient propagation while training. However, as it currently stands, these techniques are not directly applicable to data modalities where current observations depend on an entire history of previous states due to a signal's filtering properties, as common in neuroscience (and physiology more generally). Prominent examples are the blood oxygenation level dependent (BOLD) signal in functional magnetic resonance imaging (fMRI) or Ca$^{2+}$ imaging data. Such types of signals render the SSM's decoder model non-invertible, a requirement for previous TF-based methods. Here, exploiting the recent success of control techniques for training SSMs, we propose a novel algorithm that solves this problem and scales exceptionally well with model dimensionality and filter length. We demonstrate its efficiency in reconstructing dynamical systems, including their state space geometry and long-term temporal properties, from just short BOLD time series.

Integrating Multimodal Data for Joint Generative Modeling of Complex Dynamics

Reconstructing Nonlinear Dynamical Systems from Multi-Modal Time Series

Deep multi-modal learning for joint linear representation of nonlinear dynamical systems

Learning Sequential Latent Variable Models from Multimodal Time Series Data

A scalable generative model for dynamical system reconstruction from neuroimaging data

Data-driven modeling of interrelated dynamical systems

Generative Modeling of Multimodal Multi-Human Behavior

Meta-Dynamical State Space Models for Integrative Neural Data Analysis

Generative learning for nonlinear dynamics

Generative Modeling of Neural Dynamics via Latent Stochastic Differential Equations

Joint Multimodal Learning with Deep Generative Models

Learning Interpretable Hierarchical Dynamical Systems Models from Time Series Data

Learning Latent Dynamics via Invariant Decomposition and (Spatio-)Temporal Transformers

Dynamically Meaningful Latent Representations of Dynamical Systems

Latent Multimodal Functional Graphical Model Estimation

Non-intrusive model combination for learning dynamical systems

Multidimensional Approximation of Nonlinear Dynamical Systems

Deep Generative Modeling for Identification of Noisy, Non-Stationary Dynamical Systems

Inferring Brain Dynamics via Multimodal Joint Graph Representation EEG-fMRI

Learning Intuitive Physics with Multimodal Generative Models