Abstract:Nonparametric modeling approaches show very promising results in the area of system identification and control. A naturally provided model confidence is highly relevant for system-theoretical considerations to provide guarantees for application scenarios. Gaussian process regression represents one approach which provides such an indicator for the model confidence. However, this measure is only valid if the covariance function and its hyperparameters fit the underlying data generating process. In this paper, we derive an upper bound for the mean square prediction error of misspecified Gaussian process models based on a pseudo-concave optimization problem. We present application scenarios and a simulation to compare the derived upper bound with the true mean square error.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: when the covariance function and its hyper - parameters of the Gaussian Process (GP) model do not match the actual data - generating process, how to estimate the prediction error. Specifically, the author aims to derive an upper bound to quantify the maximum possible value of the Mean Square Prediction Error (MSPE) in such a mismatch situation.
### Background and Problem Description
1. **Advantages and Challenges of Non - parametric Models**
- Non - parametric models (such as Gaussian process regression) perform well in system identification and control because they do not need to assume a specific model structure and only rely on the data itself.
- However, a key issue with these models is the accuracy of estimating the model, which is very important for robust control design and stability considerations.
2. **Characteristics of the Gaussian Process Model**
- The Gaussian process not only provides the predicted mean but also the predicted variance as a measure of uncertainty.
- However, this variance as a measure of prediction error is only valid when the covariance function and its hyper - parameters of the GP model match the data - generating process.
3. **Mismatch Problem**
- In practical applications, the covariance function and its hyper - parameters are usually unknown or do not match completely.
- Therefore, the predicted variance may not correctly reflect the true model confidence, resulting in underestimation or overestimation of the prediction error.
### Main Contributions of the Paper
- **Derivation of the Upper Bound of MSPE**: The author derives an upper bound through a pseudo - concave optimization problem to estimate the mean - square prediction error of the mismatched GP model.
- **Application Scenarios**: Specific application scenarios and simulation results are provided, comparing the derived upper bound with the true mean - square error.
### Mathematical Formula Representation
The key formulas involved in the paper are as follows:
1. **Definition of Mean - Square Prediction Error**
\[
E\left[\|\Delta\|^2\right] = E\left[\|y|(x,D) - \mu(\hat{y}|x,D)\|^2\right]
\]
where \(\Delta = y|(x,D) - \mu(\hat{y}|x,D)\) is the prediction error.
2. **Expression of MSPE**
\[
E\left[\|\Delta\|^2\right] = \sum_{i = 1}^{n_y} k_i(\phi_i)- 2\hat{k}_i(\hat{\phi}_i)^T \hat{K}_i^{-1}(\hat{\phi}_i) k_i(\phi_i)+ \hat{k}_i(\hat{\phi}_i)^T \hat{K}_i^{-1} K_i(\phi_i) \hat{K}_i^{-1} \hat{k}_i(\hat{\phi}_i)
\]
3. **Derivation of the Upper Bound**
\[
E\left[\|\Delta\|^2\right] \leq n_y \bar{\alpha}(x)+ \sum_{i = 1}^{n_y} \bar{\gamma}_i(x)- 2\beta_i(x)
\]
where
\[
\bar{\alpha}(x)=\max_{j\in\{1,\ldots,z\}} \max_{\tilde{\phi}_j\in\tilde{\Phi}_j} \tilde{k}_j(\tilde{\phi}_j, x, x)
\]
\[
\beta_i(x)=\sum_{p = 1}^m \min\{h_i^p, 0\} \max_j \max_{\tilde{\phi}_j} \tilde{k}_j(\tilde{\phi}_j, x, X_{:,p})
\]
\[
\bar{\gamma}_i(x)=\sum_{p,q = 1}