Abstract:Gaussian processes~(Kriging) are interpolating data-driven models that are frequently applied in various disciplines. Often, Gaussian processes are trained on datasets and are subsequently embedded as surrogate models in optimization problems. These optimization problems are nonconvex and global optimization is desired. However, previous literature observed computational burdens limiting deterministic global optimization to Gaussian processes trained on few data points. We propose a reduced-space formulation for deterministic global optimization with trained Gaussian processes embedded. For optimization, the branch-and-bound solver branches only on the degrees of freedom and McCormick relaxations are propagated through explicit Gaussian process models. The approach also leads to significantly smaller and computationally cheaper subproblems for lower and upper bounding. To further accelerate convergence, we derive envelopes of common covariance functions for GPs and tight relaxations of acquisition functions used in Bayesian optimization including expected improvement, probability of improvement, and lower confidence bound. In total, we reduce computational time by orders of magnitude compared to state-of-the-art methods, thus overcoming previous computational burdens. We demonstrate the performance and scaling of the proposed method and apply it to Bayesian optimization with global optimization of the acquisition function and chance-constrained programming. The Gaussian process models, acquisition functions, and training scripts are available open-source within the "MeLOn - Machine Learning Models for Optimization" toolbox~(https://git.rwth-aachen.de/avt.svt/public/MeLOn).

Gaussian Processes and Bayesian Optimization for High Precision Experiments

Using Gaussian process regression for efficient parameter reconstruction

Safe model-based design of experiments using Gaussian processes

Process Quality Control Through Bayesian Optimization with Adaptive Local Convergence

Quantum Gaussian Process Regression for Bayesian Optimization

Introduction To Gaussian Process Regression In Bayesian Inverse Problems, With New ResultsOn Experimental Design For Weighted Error Measures

Bayesian Optimization using Deep Gaussian Processes

Scalable Bayesian Optimization Using Vecchia Approximations of Gaussian Processes

Bayesian Optimization with High-Dimensional Outputs

Physics Inspired Approaches To Understanding Gaussian Processes

Monte Carlo Implementation of Gaussian Process Models for Bayesian Regression and Classification

A Framework for Controlling Sources of Inaccuracy in Gaussian Process Emulation of Deterministic Computer Experiments

Global Optimization of Gaussian processes

Bayesian Modeling with Gaussian Processes using the GPstuff Toolbox

Practical Bayesian Optimization of Machine Learning Algorithms

Optimizing High-Dimensional Physics Simulations via Composite Bayesian Optimization

Upgrading from Gaussian Processes to Student's-T Processes

Robust Gaussian Process Regression method for efficient reaction pathway optimization: application to surface processes

Beyond the Mean-Field: Structured Deep Gaussian Processes Improve the Predictive Uncertainties

Optimal Designs of Gaussian Processes with Budgets for Hyperparameter Optimization

GaussianProcesses.jl: A Nonparametric Bayes package for the Julia Language