Neural filtering for Neural Network-based Models of Dynamic Systems

Parham Oveissi,Turibius Rozario,Ankit Goel

2024-09-21

Abstract:The application of neural networks in modeling dynamic systems has become prominent due to their ability to estimate complex nonlinear functions. Despite their effectiveness, neural networks face challenges in long-term predictions, where the prediction error diverges over time, thus degrading their accuracy. This paper presents a neural filter to enhance the accuracy of long-term state predictions of neural network-based models of dynamic systems. Motivated by the extended Kalman filter, the neural filter combines the neural network state predictions with the measurements from the physical system to improve the estimated state's accuracy. The neural filter's improvements in prediction accuracy are demonstrated through applications to four nonlinear dynamical systems. Numerical experiments show that the neural filter significantly improves prediction accuracy and bounds the state estimate covariance, outperforming the neural network predictions.

Machine Learning,Dynamical Systems

What problem does this paper attempt to address?

The problem that this paper attempts to solve is that when neural networks perform long - term prediction on dynamic systems, the prediction error diverges as it accumulates over time. Although neural networks can effectively estimate complex nonlinear functions and thus perform well in modeling dynamic systems, in long - term prediction, their prediction errors tend to increase over time, thereby reducing the prediction accuracy.

To solve this problem, the paper proposes a **Neural Filter** to enhance the long - term state prediction accuracy of dynamic systems based on neural network models. This neural filter draws on the idea of the Extended Kalman Filter (EKF) and improves the accuracy of the estimated state by combining the state prediction of the neural network with the measurement data of the physical system.

Specifically, the working principle of the neural filter is as follows:

1. **State prediction**: Use the neural network \( \text{NN}(x_k, u_k) \) to predict the state \( \hat{x}_{k + 1|k} \) at the next moment.
2. **State correction**: According to the actual measurement value \( y_{k + 1} \) of the physical system, correct the predicted state through the correction gain \( K_{k + 1} \) to obtain the updated state estimate \( \hat{x}_{k + 1|k + 1} \):
\[
\hat{x}_{k + 1|k + 1}=\hat{x}_{k + 1|k}+K_{k + 1}(y_{k + 1}-g(\hat{x}_{k + 1|k}))
\]
3. **Correction gain calculation**: The correction gain \( K_{k + 1} \) is given by the following formula:
\[
K_{k + 1}=P_{k + 1|k}C_{k + 1}^T(C_{k + 1}P_{k + 1|k}C_{k + 1}^T + R_{k + 1})^{-1}
\]
where \( P_{k + 1|k} \) is the prior covariance matrix, representing the uncertainty of the prediction error; \( C_{k + 1} \) is the measurement matrix, representing the Jacobian matrix of the measurement function \( g \); \( R_{k + 1} \) is the covariance matrix of the measurement noise.

In this way, the neural filter can continuously use the actual measurement data to correct the prediction results during the prediction process, thereby effectively suppressing the accumulation of errors and improving the accuracy of long - term prediction.

The paper verifies the effectiveness of the neural filter through numerical experiments on four nonlinear dynamic systems (simple pendulum, Van der Pol oscillator, Lorenz system, and double pendulum system). The results show that the neural filter significantly improves the prediction accuracy and can maintain the boundedness of the state - estimate covariance, which is superior to the pure neural network prediction.