Abstract:Representation learning for high-dimensional, complex physical systems aims to identify a low-dimensional intrinsic latent space, which is crucial for reduced-order modeling and modal analysis. To overcome the well-known Kolmogorov barrier, deep autoencoders (AEs) have been introduced in recent years, but they often suffer from poor convergence behavior as the rank of the latent space increases. To address this issue, we propose the learnable weighted hybrid autoencoder, a hybrid approach that combines the strengths of singular value decomposition (SVD) with deep autoencoders through a learnable weighted framework. We find that the introduction of learnable weighting parameters is essential - without them, the resulting model would either collapse into a standard POD or fail to exhibit the desired convergence behavior. Additionally, we empirically find that our trained model has a sharpness thousands of times smaller compared to other models. Our experiments on classical chaotic PDE systems, including the 1D Kuramoto-Sivashinsky and forced isotropic turbulence datasets, demonstrate that our approach significantly improves generalization performance compared to several competing methods, paving the way for robust representation learning of high-dimensional, complex physical systems.

What problem does this paper attempt to address?

The problems that this paper attempts to solve are: How to overcome the limitations of traditional dimension - reduction methods (such as POD) and existing deep auto - encoders (AE) when dealing with high - dimensional complex physical systems, especially the problem of the poor convergence behavior of deep auto - encoders when the rank of the latent space increases. Specifically: 1. **Dimension - reduction problem of high - dimensional complex systems**: - The author points out that for high - dimensional complex physical systems (such as turbulence in fluid mechanics), traditional linear dimension - reduction techniques (such as POD) can effectively reduce dimensions, but perform poorly in capturing nonlinear features. - Deep auto - encoders (AE) can capture nonlinear features, but as the rank of the latent space increases, their convergence performance deteriorates, resulting in difficult - to - train models and insufficient generalization ability. 2. **SVD - like convergence problem**: - Existing deep auto - encoders often cannot converge as quickly as singular value decomposition (SVD) when dealing with high - dimensional data. SVD is a classic linear dimension - reduction method with good convergence characteristics, but its expressive ability for nonlinear systems is limited. 3. **Improving the robustness and generalization ability of the model**: - Deep auto - encoders are prone to over - fitting when facing noisy data, resulting in poor generalization performance. Therefore, how to improve the robustness of the model to noise is also an important issue. To solve these problems, the author proposes a new weighted hybrid auto - encoder framework, which combines the advantages of SVD and deep auto - encoders and enhances the performance of the model by introducing learnable weight parameters. This framework not only exhibits SVD - like convergence characteristics but also can maintain good generalization ability and robustness in high - rank cases. In summary, the main objective of this paper is to solve the convergence and generalization ability problems of existing methods when dealing with high - dimensional complex physical systems by proposing a new deep auto - encoder framework, thereby achieving more effective dimension - reduction and better model performance.

Deep Autoencoder with SVD-Like Convergence and Flat Minima

Learning a Reduced Basis of Dynamical Systems using an Autoencoder

Autoencoders for discovering manifold dimension and coordinates in data from complex dynamical systems

Latent Space Characterization of Autoencoder Variants

Stacked Wasserstein Autoencoder

Systematic Development of a New Variational Autoencoder Model Based on Uncertain Data for Monitoring Nonlinear Processes

Symplectic encoders for physics-constrained variational dynamics inference

Solving deep-learning density functional theory via variational autoencoders

Bridging Autoencoders and Dynamic Mode Decomposition for Reduced-order Modeling and Control of PDEs

Structure-preserving model reduction of Hamiltonian systems by learning a symplectic autoencoder

Deep autoencoders for physics-constrained data-driven nonlinear materials modeling

Deep convolutional recurrent autoencoders for learning low-dimensional feature dynamics of fluid systems

Learning Nonlinear Projections for Reduced-Order Modeling of Dynamical Systems using Constrained Autoencoders

Hidden Talents of the Variational Autoencoder

Learning Latent Dynamics via Invariant Decomposition and (Spatio-)Temporal Transformers

Unsupervised Machine Learning to Teach Fluid Dynamicists to Think in 15 Dimensions

Deep Kernel Methods Learn Better: From Cards to Process Optimization

$Φ$-DVAE: Physics-Informed Dynamical Variational Autoencoders for Unstructured Data Assimilation

Variational Encoder-Decoders for Learning Latent Representations of Physical Systems

Understanding Latent Timescales in Neural Ordinary Differential Equation Models for Advection-Dominated Dynamical Systems

The Deep Kernelized Autoencoder