Abstract:Evolutionary partial differential equations play a crucial role in many areas of science and engineering. Spatial discretization of these equations leads to a system of ordinary differential equations which can then be solved by numerical time integration. Such a system is often of very high dimension, making the simulation very time consuming. One way to reduce the computational cost is to approximate the large system by a low-dimensional model using a model reduction approach. This master thesis deals with structure-preserving model reduction of Hamiltonian systems by using machine learning techniques. We discuss a nonlinear approach based on the construction of an encoder-decoder pair that minimizes the approximation error and satisfies symplectic constraints to guarantee the preservation of the structure inherent in Hamiltonian systems. More specifically, we study an autoencoder network that learns a symplectic encoder-decoder pair. Symplecticity poses some additional difficulties, as we need to ensure this structure in each network layer. Since these symplectic constraints are described by the (symplectic) Stiefel manifold, we use manifold optimization techniques to ensure the symplecticity of the encoder and decoder. A particular challenge is to adapt the ADAM optimizer to the manifold structure. We present a modified ADAM optimizer that works directly on the Stiefel manifold and compare it to the existing approach based on homogeneous spaces. In addition, we propose several modifications to the network and training setup that significantly improve the performance and accuracy of the autoencoder. Finally, we numerically validate the modified optimizer and different learning configurations on two Hamiltonian systems, the 1D wave equation and the sine-Gordon equation, and demonstrate the improved accuracy and computational efficiency of the presented learning algorithms.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: how to perform structure - preserving model reduction on Hamiltonian systems through machine learning techniques, especially by using symplectic autoencoders. Specifically, the paper focuses on ensuring that the reduced - order model can retain the symplectic structure of the original high - dimensional system while reducing the computational cost. ### Problem Background Hamiltonian systems play an important role in many scientific and engineering fields. However, the numerical simulations of these systems are usually very time - consuming because their spatial discretization leads to high - dimensional ordinary differential equations (ODEs). To solve this problem, the high - dimensional system can be approximated as a low - dimensional model through the model reduction method, thereby reducing the computational cost. ### Core Problems of the Paper 1. **Structure Preservation**: How to ensure that the reduced - order model still maintains the symplectic structure of the Hamiltonian system? This is crucial because the symplectic structure guarantees that the physical properties of the system (such as energy conservation) are preserved. 2. **Application of Machine Learning**: How to use machine learning techniques, especially the autoencoder network in deep learning, to achieve this structure - preserving model reduction? 3. **Optimization Challenges**: Since it is necessary to ensure that each layer of the network satisfies the symplectic constraints, this brings additional difficulties to the optimization process. Specifically, these symplectic constraints are described by the symplectic Stiefel manifold, so manifold optimization techniques need to be used. ### Solutions The paper proposes a method based on symplectic autoencoders, which solves the problem through the following steps: 1. **Constructing Encoder - Decoder Pairs**: Design a nonlinear model reduction method. Minimize the approximation error and ensure that the symplectic constraints are met by constructing encoder - decoder pairs. 2. **Manifold Optimization**: In order to ensure that each layer of the encoder and decoder maintains the symplectic structure, use manifold optimization techniques, especially optimize on the symplectic Stiefel manifold. 3. **Improving the Optimizer**: Propose an improved ADAM optimizer so that it can work directly on the symplectic Stiefel manifold, and compare it with the existing methods based on homogeneous spaces. 4. **Numerical Verification**: Through numerical experiments on two Hamiltonian systems (1D wave equation and Sine - Gordon equation), verify the effectiveness and computational efficiency of the improved optimizer and different learning configurations. ### Summary The main objective of the paper is to explore how to use symplectic autoencoders to learn encoder - decoder pairs to achieve structure - preserving model reduction of Hamiltonian systems. By introducing new manifold optimization steps and an improved optimizer, the paper shows how to significantly improve the accuracy and computational efficiency of the reduced - order model while maintaining the symplectic structure.

Structure-preserving model reduction of Hamiltonian systems by learning a symplectic autoencoder

Symplectic Autoencoders for Model Reduction of Hamiltonian Systems

Symplectic Model Reduction of Hamiltonian Systems on Nonlinear Manifolds

Learning Nonlinear Projections for Reduced-Order Modeling of Dynamical Systems using Constrained Autoencoders

Learning Generalized Hamiltonians using fully Symplectic Mappings

Neural Autoencoder-Based Structure-Preserving Model Order Reduction and Control Design for High-Dimensional Physical Systems

Symplectic Learning for Hamiltonian Neural Networks

Hamiltonian reduction using a convolutional auto-encoder coupled to an Hamiltonian neural network

Learning a Reduced Basis of Dynamical Systems using an Autoencoder

Data-driven Prediction of General Hamiltonian Dynamics via Learning Exactly-Symplectic Maps

Non-linear Manifold Reduced-Order Models with Convolutional Autoencoders and Reduced Over-Collocation Method

Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders

Structure-preserving learning for multi-symplectic PDEs

Hamiltonian neural networks for solving equations of motion

Applications of Machine Learning to Modelling and Analysing Dynamical Systems

Data-driven structure-preserving model reduction for stochastic Hamiltonian systems

Understanding Latent Timescales in Neural Ordinary Differential Equation Models for Advection-Dominated Dynamical Systems

Symplectic Neural Networks Based on Dynamical Systems

Hamiltonian Matching for Symplectic Neural Integrators

Deep convolutional recurrent autoencoders for learning low-dimensional feature dynamics of fluid systems

Symplectic model reduction of Hamiltonian systems using data-driven quadratic manifolds