Abstract:Neural Cellular Automata (NCA) are a powerful combination of machine learning and mechanistic modelling. We train NCA to learn complex dynamics from time series of images and PDE trajectories. Our method is designed to identify underlying local rules that govern large scale dynamic emergent behaviours. Previous work on NCA focuses on learning rules that give stationary emergent structures. We extend NCA to capture both transient and stable structures within the same system, as well as learning rules that capture the dynamics of Turing pattern formation in nonlinear Partial Differential Equations (PDEs). We demonstrate that NCA can generalise very well beyond their PDE training data, we show how to constrain NCA to respect given symmetries, and we explore the effects of associated hyperparameters on model performance and stability. Being able to learn arbitrary dynamics gives NCA great potential as a data driven modelling framework, especially for modelling biological pattern formation. Pattern formation is ubiquitous in biological systems, across many length and time scales—from vegetation stripes in deserts, to the spots of a leopard's skin, and the fine detail of stem cell differentiation in an embryo. While many simple rules that create complex patterns are known, reverse engineering the mechanisms responsible from an observed pattern has generally remained a difficult problem. In this work we build on the connections between machine learning, cellular automata, and partial differential equations, to create models that can learn the underlying mechanisms that yield any desired emergent pattern. To do this we consider Neural Cellular Automata (a mix of neural networks and cellular automata). We describe how they are built and trained, and we show that they are capable of reproducing a wide range of complex emergent behaviours. Previous work focuses on learning stationary patterns, but we focus on patterns that evolve in time. We show these models can learn classic Turing patterns (biochemical models of spot and stripe formation), as well as learning to morph through an arbitrary sequence of images. We believe this hybrid between machine learning and mechanistic modelling has great potential for modelling biological growth, regeneration, and pattern formation.

TRENDy: Temporal Regression of Effective Non-linear Dynamics

Beyond Predictions in Neural ODEs: Identification and Interventions.

Sparsity in Continuous-Depth Neural Networks

Let's do the time-warp-attend: Learning topological invariants of dynamical systems

Learning the intrinsic dynamics of spatio-temporal processes through Latent Dynamics Networks

TREND: TempoRal Event and Node Dynamics for Graph Representation Learning

Neural Persistence Dynamics

Learning spatio-temporal patterns with Neural Cellular Automata

A Deep Learning Approach for Predicting Spatiotemporal Dynamics From Sparsely Observed Data

Unraveled Multilevel Transformation Networks for Predicting Sparsely-Observed Spatiotemporal Dynamics

Modeling Randomly Observed Spatiotemporal Dynamical Systems

Data-driven prediction of large-scale spatiotemporal chaos with distributed low-dimensional models

Global dynamical structures from infinitesimal data

Latent Dynamics Networks (LDNets): learning the intrinsic dynamics of spatio-temporal processes

Unravelled multilevel transformation networks for predicting sparsely observed spatio-temporal dynamics

Data-driven spatiotemporal modeling for structural dynamics on irregular domains by stochastic dependency neural estimation

Fourier neural operators for spatiotemporal dynamics in two-dimensional turbulence

Learning continuous network emerging dynamics from scarce observations via data-adaptive stochastic processes

Itrendrnn: an Interpretable Trend-Aware RNN for Meteorological Spatiotemporal Prediction

Learning Latent Dynamics via Invariant Decomposition and (Spatio-)Temporal Transformers

A Spatial Filtering Approach to Biological Patterning