Latent Dynamics Networks (LDNets): learning the intrinsic dynamics of spatio-temporal processes

Francesco Regazzoni,Stefano Pagani,Matteo Salvador,Luca Dede',Alfio Quarteroni
2023-04-29
Abstract:Predicting the evolution of systems that exhibit spatio-temporal dynamics in response to external stimuli is a key enabling technology fostering scientific innovation. Traditional equations-based approaches leverage first principles to yield predictions through the numerical approximation of high-dimensional systems of differential equations, thus calling for large-scale parallel computing platforms and requiring large computational costs. Data-driven approaches, instead, enable the description of systems evolution in low-dimensional latent spaces, by leveraging dimensionality reduction and deep learning algorithms. We propose a novel architecture, named Latent Dynamics Network (LDNet), which is able to discover low-dimensional intrinsic dynamics of possibly non-Markovian dynamical systems, thus predicting the time evolution of space-dependent fields in response to external inputs. Unlike popular approaches, in which the latent representation of the solution manifold is learned by means of auto-encoders that map a high-dimensional discretization of the system state into itself, LDNets automatically discover a low-dimensional manifold while learning the latent dynamics, without ever operating in the high-dimensional space. Furthermore, LDNets are meshless algorithms that do not reconstruct the output on a predetermined grid of points, but rather at any point of the domain, thus enabling weight-sharing across query-points. These features make LDNets lightweight and easy-to-train, with excellent accuracy and generalization properties, even in time-extrapolation regimes. We validate our method on several test cases and we show that, for a challenging highly-nonlinear problem, LDNets outperform state-of-the-art methods in terms of accuracy (normalized error 5 times smaller), by employing a dramatically smaller number of trainable parameters (more than 10 times fewer).
Machine Learning,Numerical Analysis
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to predict the evolution of spatio - temporal dynamic systems when responding to external stimuli. Traditional equation - based methods use first principles to generate predictions through numerical approximation of high - dimensional differential equation systems, which require large - scale parallel computing platforms and are extremely computationally costly. Data - driven methods, on the other hand, describe system evolution in a low - dimensional latent space through dimension reduction and deep - learning algorithms, thus reducing the demand for high - computing resources. The paper proposes a new architecture, called Latent Dynamics Network (LDNet), which can discover the low - dimensional intrinsic dynamics of potentially non - Markovian dynamic systems, thereby predicting the time evolution of spatially - dependent fields under external inputs. Unlike popular auto - encoder methods, LDNet automatically discovers low - dimensional manifolds and learns latent dynamics without operating in high - dimensional space. In addition, LDNet is a mesh - free algorithm that does not reconstruct the output on a predetermined point grid, but rather at any point in the domain, enabling weight sharing between query points. These features make LDNet lightweight and easy to train, with extremely high accuracy and generalization performance, even in the case of time extrapolation. Specifically, the challenges mentioned in the paper include: - **Computational Cost**: Traditional methods require a large amount of computing resources, especially in multi - query scenarios. - **Model Complexity**: Building a model requires a deep understanding of physical processes, using physical first principles or empirical rules, and translating them into mathematical terms. - **Parameter Uncertainty**: The parameter values, boundary conditions, and initial conditions required by the model are usually unknown, increasing the intrinsic dimension of the solution space. - **Data - Driven Modeling**: In recent years, the emergence of data - driven modeling methods has provided a new way to overcome the limitations of traditional physical models, either as an alternative or in synergy with traditional methods. The paper verifies the effectiveness of LDNet through multiple test cases and shows that when dealing with highly nonlinear problems, LDNet is significantly superior to existing methods in terms of accuracy (with a 5 - fold smaller normalized error), while also using a significantly reduced number of trainable parameters (more than 10 - fold).