Neural Ordinary Differential Equations for Model Order Reduction of Stiff Systems

Matteo Caldana,Jan S. Hesthaven
2024-08-12
Abstract:Neural Ordinary Differential Equations (ODEs) represent a significant advancement at the intersection of machine learning and dynamical systems, offering a continuous-time analog to discrete neural networks. Despite their promise, deploying neural ODEs in practical applications often encounters the challenge of stiffness, a condition where rapid variations in some components of the solution demand prohibitively small time steps for explicit solvers. This work addresses the stiffness issue when employing neural ODEs for model order reduction by introducing a suitable reparametrization in time. The considered map is data-driven and it is induced by the adaptive time-stepping of an implicit solver on a reference solution. We show the map produces a nonstiff system that can be cheaply solved with an explicit time integration scheme. The original, stiff, time dynamic is recovered by means of a map learnt by a neural network that connects the state space to the time reparametrization. We validate our method through extensive experiments, demonstrating improvements in efficiency for the neural ODE inference while maintaining robustness and accuracy. The neural network model also showcases good generalization properties for times beyond the training data.
Numerical Analysis
What problem does this paper attempt to address?
The paper attempts to address the issue of stiffness encountered when using Neural Ordinary Differential Equations (ODEs) for constructing Reduced Order Models (ROMs). Specifically, when dealing with scenarios where fast-varying and slow-varying components coexist, traditional explicit solvers become inefficient or even unstable. The paper proposes a method to reduce system stiffness through appropriate time reparameterization, allowing the use of explicit time integration schemes to efficiently solve originally stiff systems. The main contributions include: 1. Proposing a data-driven method that reduces system stiffness by utilizing time reparameterization induced by the time stepping of implicit solvers. 2. Learning the mapping from state space to time reparameterization through neural networks, thereby mapping the solution of the non-stiff system back to the original stiff dynamics. 3. Validating the effectiveness and speed of the proposed method on a series of benchmark problems and comparing it with existing stiff ODE solvers. This method not only improves the efficiency of neural ODE inference but also maintains its robustness and accuracy, and demonstrates good generalization performance beyond the time range of the training data.