Abstract:The neural dynamics underlying brain activity are critical to understanding cognitive processes and mental disorders. However, current voxel-based whole-brain dimensionality reduction techniques fall short of capturing these dynamics, producing latent timeseries that inadequately relate to behavioral tasks. To address this issue, we introduce a novel approach to learning low-dimensional approximations of neural dynamics by using a sequential variational autoencoder (SVAE) that represents the latent dynamical system via a neural ordinary differential equation (NODE). Importantly, our method finds smooth dynamics that can predict cognitive processes with accuracy higher than classical methods. Our method also shows improved spatial localization to task-relevant brain regions and identifies well-known structures such as the motor homunculus from fMRI motor task recordings. We also find that non-linear projections to the latent space enhance performance for specific tasks, offering a promising direction for future research. We evaluate our approach on various task-fMRI datasets, including motor, working memory, and relational processing tasks, and demonstrate that it outperforms widely used dimensionality reduction techniques in how well the latent timeseries relates to behavioral sub-tasks, such as left-hand or right-hand tapping. Additionally, we replace the NODE with a recurrent neural network (RNN) and compare the two approaches to understand the importance of explicitly learning a dynamical system. Lastly, we analyze the robustness of the learned dynamical systems themselves and find that their fixed points are robust across seeds, highlighting our method's potential for the analysis of cognitive processes as dynamical systems.

NewtonianVAE: Proportional Control and Goal Identification from Pixels via Physical Latent Spaces

Learn Proportional Derivative Controllable Latent Space from Pixels

ControlVAE: Model-Based Learning of Generative Controllers for Physics-Based Characters

Tracking control of latent dynamic systems with application to spacecraft attitude control

Learning deep dynamical models from image pixels

Feedback from Pixels: Output Regulation via Learning-Based Scene View Synthesis

PcLast: Discovering Plannable Continuous Latent States

Character Controllers Using Motion VAEs

LVD-NMPC: A Learning-based Vision Dynamics Approach to Nonlinear Model Predictive Control for Autonomous Vehicles

Learning low-dimensional dynamics from whole-brain data improves task capture

Learning Sequential Latent Variable Models from Multimodal Time Series Data

Interpretable Representation Learning from Videos using Nonlinear Priors

Learning Nonlinear Projections for Reduced-Order Modeling of Dynamical Systems using Constrained Autoencoders

Back to Newton's Laws: Learning Vision-based Agile Flight via Differentiable Physics

Can Direct Latent Model Learning Solve Linear Quadratic Gaussian Control?

Towards Learning Controllable Representations of Physical Systems

Identifiable Representation and Model Learning for Latent Dynamic Systems

VMP: Versatile Motion Priors for Robustly Tracking Motion on Physical Characters

Visual Imitation Learning of Non-Prehensile Manipulation Tasks with Dynamics-Supervised Models

Real-Time Variational Method for Learning Neural Trajectory and its Dynamics

Bayesian Optimization in Variational Latent Spaces with Dynamic Compression