Abstract:The Gaussian process (GP) regression model is a widely employed surrogate modeling technique for computer experiments, offering precise predictions and statistical inference for the computer simulators that generate experimental data. Estimation and inference for GP can be performed in both frequentist and Bayesian frameworks. In this chapter, we construct the GP model through variational inference, particularly employing the recently introduced energetic variational inference method by Wang et al. (2021). Adhering to the GP model assumptions, we derive posterior distributions for its parameters. The energetic variational inference approach bridges the Bayesian sampling and optimization and enables approximation of the posterior distributions and identification of the posterior mode. By incorporating a normal prior on the mean component of the GP model, we also apply shrinkage estimation to the parameters, facilitating mean function variable selection. To showcase the effectiveness of our proposed GP model, we present results from three benchmark examples.

What problem does this paper attempt to address?

### Problems the paper attempts to solve This paper aims to improve the application of Gaussian Process Regression (GP) in computer experiments by introducing a new variational inference method - Energetic Variational Inference (EVI). Specifically, the paper addresses the following problems: 1. **Parameter estimation and inference**: - Gaussian Process Regression models usually need to estimate and infer model parameters. In the traditional Bayesian framework, the commonly used method is Markov Chain Monte Carlo (MCMC) sampling. However, the MCMC method is computationally complex and time - consuming. - The paper proposes using the Energetic Variational Inference (EVI) method to replace MCMC sampling to more efficiently generate samples of the posterior distribution or calculate the Maximum A Posteriori (MAP) estimate. 2. **Model selection and sparsity**: - In Gaussian Process Regression, the selection of the mean function and sparsity are important issues. The paper achieves variable selection of the mean function by introducing a normal prior, thereby achieving the purpose of model sparsification. - Through \( l_2 \)-regularization, the complexity of the model can be reduced and the prediction accuracy can be improved. 3. **Computational efficiency**: - The traditional MCMC method has low computational efficiency when dealing with large - scale data. The EVI method combines the particle method and variational inference to provide a more efficient computational tool. - The paper shows the performance of the EVI method in multiple benchmark tests, proving its superiority in computational efficiency and prediction accuracy. 4. **Theoretical basis**: - The paper details the theoretical basis of the EVI method, including the law of energy dissipation, the principle of least action, and the principle of maximum dissipation. - Through these theories, the EVI method can effectively approximate the posterior distribution while maintaining numerical stability. ### Main contributions - **Proposing the EVI - GP method**: Combining Energetic Variational Inference and Gaussian Process Regression, providing a new computational tool for generating samples of the posterior distribution or calculating MAP estimates. - **Improving computational efficiency**: Compared with the traditional MCMC method, the EVI method has significant advantages in computational efficiency, especially when dealing with large - scale data. - **Model sparsification**: By introducing normal prior and \( l_2 \)-regularization, variable selection of the mean function is achieved, improving the sparsity of the model and prediction accuracy. - **Combining theory and practice**: Not only providing theoretical support, but also verifying the effectiveness of the EVI - GP method in multiple benchmark tests. ### Conclusion By introducing the EVI method, the paper successfully solves some key problems of Gaussian Process Regression in computer experiments, such as parameter estimation, model selection, and computational efficiency. The performance of the EVI - GP method in multiple benchmark tests is better than other existing methods, proving its potential and value in practical applications.

Energetic Variational Gaussian Process Regression for Computer Experiments

Computer emulation with non-stationary Gaussian processes

A Fast GP Regression Method Using Banded Sparsification of Inverse Covariance

Asynchronous Distributed Variational Gaussian Process for Regression.

Robust and Scalable Gaussian Process Regression and Its Applications

Variable Sigma Gaussian Processes: an Expectation Propagation Perspective

ProSpar-GP: scalable Gaussian process modeling with massive non-stationary datasets

Robust and Conjugate Gaussian Process Regression

Mixture of robust Gaussian processes and its hard-cut EM algorithm with variational bounding approximation

EzGP: Easy-to-Interpret Gaussian Process Models for Computer Experiments with Both Quantitative and Qualitative Factors

Variational Gaussian Processes: A Functional Analysis View

Leveraging the Nugget Parameter for Efficient Gaussian Process Modeling

Stein Variational Gaussian Processes

Towards Scalable Gaussian Process Modeling

Modulating Scalable Gaussian Processes for Expressive Statistical Learning

Amortized Variational Inference for Deep Gaussian Processes

Distributed Variational Inference in Sparse Gaussian Process Regression and Latent Variable Models

Latent Gaussian Process Regression

Response Probability Distribution Estimation of Expensive Computer Simulators: A Bayesian Active Learning Perspective Using Gaussian Process Regression

Non-stationary Gaussian Process Surrogates

Cautious Model Predictive Control Using Gaussian Process Regression