Abstract:Absence of sufficiently high-quality data often poses a key challenge in data-driven modeling of high-dimensional spatio-temporal dynamical systems. Koopman Autoencoders (KAEs) harness the expressivity of deep neural networks (DNNs), the dimension reduction capabilities of autoencoders, and the spectral properties of the Koopman operator to learn a reduced-order feature space with simpler, linear dynamics. However, the effectiveness of KAEs is hindered by limited and noisy training datasets, leading to poor generalizability. To address this, we introduce the Temporally-Consistent Koopman Autoencoder (tcKAE), designed to generate accurate long-term predictions even with constrained and noisy training data. This is achieved through a consistency regularization term that enforces prediction coherence across different time steps, thus enhancing the robustness and generalizability of tcKAE over existing models. We provide analytical justification for this approach based on Koopman spectral theory and empirically demonstrate tcKAE's superior performance over state-of-the-art KAE models across a variety of test cases, including simple pendulum oscillations, kinetic plasmas, fluid flows, and sea surface temperature data.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper aims to address the challenges of data - driven modeling due to insufficient data quality in high - dimensional spatio - temporal dynamical systems. Specifically, Koopman Autoencoders (KAEs) utilize the expressive power of deep neural networks (DNNs), the dimension - reduction ability of autoencoders, and the spectral properties of the Koopman operator to learn a reduced - order feature space with simpler linear dynamics. However, existing KAE methods perform poorly when dealing with limited and noisy training data, resulting in poor generalization ability. To solve these problems, the authors introduce a new model - **Temporally - Consistent Koopman Autoencoder (tcKAE)**. This model improves the robustness and generalization ability of the model by introducing a **consistency regularization term** to ensure the consistency of predictions between different time steps, enabling it to generate accurate long - term predictions even in the case of limited data and high noise. ### Main contributions 1. **Improved generalization ability**: tcKAE enhances the generalization ability of the model through the consistency regularization term, especially in cases where the amount of data is limited or there is noise. 2. **Consistency in multi - step prediction**: Compared with traditional KAEs, tcKAE not only focuses on single - step prediction errors but also ensures the consistency between multi - step predictions through consistency constraints. 3. **No need to rely on labeled data**: tcKAE does not rely on labeled data for the calculation of multi - step prediction losses. Instead, it achieves consistency constraints by comparing the prediction results of different initial time instances, which reduces the need for a large amount of labeled data. 4. **Theoretical analysis and experimental verification**: Based on Koopman spectral theory, theoretical basis is provided, and the superior performance of tcKAE is verified through multiple test cases (such as simple pendulum, plasma dynamics, fluid flow, and sea - surface temperature data). ### Key formulas 1. **Definition of Koopman operator**: \[ K\psi(\cdot)=\psi\circ f(\cdot) \] where \(K\) is an infinite - dimensional linear operator, \(\psi\) is an observation function, and \(f\) is the state - transition mapping of the system. 2. **Conditions for Koopman invariant subspace**: \[ \Psi_e(x_{n + \kappa})=K^{\kappa}\Psi_e(x_n) \] where \(\Psi_e\) is an encoder function, \(K\) is a finite - dimensional linear operator, and \(\kappa\) represents the number of time steps. 3. **Consistency loss function**: \[ L_{tc}=\frac{1}{2\kappa_m}\sum_{\kappa = 1}^{\kappa_m}L_\kappa,\quad L_\kappa=\frac{1}{M - 1}\sum_{p = 1}^{M - 1}\frac{1}{p}\sum_{q = 1}^{p}\|z^{(\kappa)}_{n+\kappa + p}-z^{(\kappa + q)}_{n+\kappa + p}\|_2^2 \] 4. **Total loss function**: \[ L_{tot}=\gamma_{id}L_{id}+\gamma_{fwd}L_{fwd}+\gamma_{bwd}L_{bwd}+\gamma_{con}L_{con}+\gamma_{tc}L_{tc} \] Through these improvements, tcKAE can provide more accurate and stable long - term predictions in cases where the amount of data is limited and there is high noise.

Temporally-Consistent Koopman Autoencoders for Forecasting Dynamical Systems

Tracking and forecasting oscillatory data streams using Koopman autoencoders and Kalman filtering

Koopman Invertible Autoencoder: Leveraging Forward and Backward Dynamics for Temporal Modeling

Forecasting Dynamics by an Incomplete Equation of Motion and an Auto-Encoder Koopman Operator

Autoencoding for the "Good Dictionary" of eigenpairs of the Koopman operator

Koopman dynamic-oriented deep learning for invariant subspace identification and full-state prediction of complex systems

Autoencoding for the 'Good Dictionary' of eigen pairs of the Koopman Operator

A Deep Koopman Operator‐based Modelling Approach for Long‐term Prediction of Dynamics with Pixel‐level Measurements

KODA: A Data-Driven Recursive Model for Time Series Forecasting and Data Assimilation using Koopman Operators

Neural Koopman prior for data assimilation

Koopman AutoEncoder via Singular Value Decomposition for Data-Driven Long-Term Prediction

Linearly-Recurrent Autoencoder Networks for Learning Dynamics

Eigenvalue initialisation and regularisation for Koopman autoencoders

Linearly Recurrent Autoencoder Networks for Learning Dynamics

Enhancing predictive capabilities in data-driven dynamical modeling with automatic differentiation: Koopman and neural ODE approaches

Deep Probabilistic Koopman: Long-term time-series forecasting under periodic uncertainties

From Fourier to Koopman: Spectral Methods for Long-term Time Series Prediction

Learning Deep Neural Network Representations for Koopman Operators of Nonlinear Dynamical Systems

Meta Koopman Decomposition for Time Series Forecasting under Temporal Distribution Shifts

Course Correcting Koopman Representations

Featurizing Koopman Mode Decomposition For Robust Forecasting