N. Benjamin Erichson,Michael Muehlebach,Michael W. Mahoney
Abstract:In addition to providing high-profile successes in computer vision and natural language processing, neural networks also provide an emerging set of techniques for scientific problems. Such data-driven models, however, typically ignore physical insights from the scientific system under consideration. Among other things, a physics-informed model formulation should encode some degree of stability or robustness or well-conditioning (in that a small change of the input will not lead to drastic changes in the output), characteristic of the underlying scientific problem. We investigate whether it is possible to include physics-informed prior knowledge for improving the model quality (e.g., generalization performance, sensitivity to parameter tuning, or robustness in the presence of noisy data). To that extent, we focus on the stability of an equilibrium, one of the most basic properties a dynamic system can have, via the lens of Lyapunov analysis. For the prototypical problem of fluid flow prediction, we show that models preserving Lyapunov stability improve the generalization error and reduce the prediction uncertainty.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is how to incorporate physical knowledge in fluid flow prediction to improve the stability and generalization performance of the model. Specifically, the paper explores how to improve the Autoencoder model by introducing Lyapunov stability, enabling it to better capture the stability characteristics of dynamic systems, thereby reducing prediction uncertainty when predicting fluid flow and enhancing the generalization ability of the model.
### Background and Motivation of the Paper
In many scientific and engineering problems, such as physical fluid flow, atmosphere - ocean interaction, neurophysiological responses, and economic and financial time series, etc., they can be modeled as dynamic systems. These systems usually exhibit rich multi - scale structures, so it has become increasingly important to use data - driven methods (especially deep learning and neural network techniques) to model these systems. However, existing data - driven models often ignore the physical knowledge behind the systems, which may cause the models to become fragile when facing noisy data, small training sets or multiple hyper - parameters.
### Main Contributions of the Paper
1. **Introduction of Physical Information**: The paper proposes a method to enhance the stability of the Autoencoder model by introducing Lyapunov stability as prior knowledge. Specifically, the author designs a stability - enhanced regularization term, so that the model can maintain the stability characteristics of the system during the learning process.
2. **Improvement of Generalization Performance**: By introducing physical information, the generalization performance of the model has been significantly improved. Experimental results show that the physically - informed model has better stability and lower prediction uncertainty when predicting fluid flow.
3. **Verification in Practical Applications**: The paper verifies the effectiveness of the method through simulation and real - world experiments. The experiments include the fluid flow behind a periodic cylinder and the sea surface temperature prediction in the Gulf of Mexico. The results show that the physically - informed model exhibits better generalization performance under multiple hyper - parameter configurations.
### Mathematical and Physical Background
- **Dynamic System Model**:
\[
x_{t + 1}=A(x_t)+\eta_t, \quad t = 0,1,2,\ldots
\]
where \(x_t\) represents the state of the system at time \(t\), \(A\) is a mapping function, and \(\eta_t\) is a small perturbation.
- **Linear Approximation**:
\[
x_{t + 1}=A x_t+\eta_t, \quad t = 0,1,2,\ldots
\]
where \(A\) is a linear mapping.
- **Observation Model**:
\[
y_t=G(x_t)+\xi_t, \quad t = 0,1,2,\ldots,T
\]
where \(G\) is a mapping function that maps the state \(x_t\) to the observation space \(Y\), and \(\xi_t\) represents the measurement error.
- **Lyapunov Stability**:
- **Definition**: The origin of the system is stable if for any \(\epsilon>0\), there exists \(\delta>0\) such that when \(|x_0|<\delta\), we have \(|A^k(x_0)|<\epsilon\) for all \(k = 1,2,3,\ldots\)
- **Asymptotic Stability**: If the origin of the system is not only stable but also satisfies \(\lim_{k\rightarrow\infty}A^k(x_0) = 0\), then it is called asymptotically stable.
- **Lyapunov Function**:
- There exists a continuous function \(V:\mathbb{R}^n\rightarrow\mathbb{R}\) such that \(V(0) = 0\), \(V(x)>0\) for all \(x\neq0\), and \(V(A(x))-V(x)\leq0\) for all \(x\neq0\) and \(|x|<r\).