Abstract:Gaussian processes~(Kriging) are interpolating data-driven models that are frequently applied in various disciplines. Often, Gaussian processes are trained on datasets and are subsequently embedded as surrogate models in optimization problems. These optimization problems are nonconvex and global optimization is desired. However, previous literature observed computational burdens limiting deterministic global optimization to Gaussian processes trained on few data points. We propose a reduced-space formulation for deterministic global optimization with trained Gaussian processes embedded. For optimization, the branch-and-bound solver branches only on the degrees of freedom and McCormick relaxations are propagated through explicit Gaussian process models. The approach also leads to significantly smaller and computationally cheaper subproblems for lower and upper bounding. To further accelerate convergence, we derive envelopes of common covariance functions for GPs and tight relaxations of acquisition functions used in Bayesian optimization including expected improvement, probability of improvement, and lower confidence bound. In total, we reduce computational time by orders of magnitude compared to state-of-the-art methods, thus overcoming previous computational burdens. We demonstrate the performance and scaling of the proposed method and apply it to Bayesian optimization with global optimization of the acquisition function and chance-constrained programming. The Gaussian process models, acquisition functions, and training scripts are available open-source within the "MeLOn - Machine Learning Models for Optimization" toolbox~(https://git.rwth-aachen.de/avt.svt/public/MeLOn).

Nonlinear Planning Model With a Gaussian Criterion of Optimization (Gaussian Programming Model)

Nonlinear Probabilistic Monitoring Based on the Gaussian Process Latent Variable Model

Gaussian Process Planning with Lipschitz Continuous Reward Functions: Towards Unifying Bayesian Optimization, Active Learning, and Beyond

Bayesian Optimization using Deep Gaussian Processes

A Novel Gaussian Process Surrogate Model with Expected Prediction Error for Optimization under Constraints

Global Optimization of Gaussian Process Acquisition Functions Using a Piecewise-Linear Kernel Approximation

Estimating linear covariance models with numerical nonlinear algebra

A Functional Model Method for Nonconvex Nonsmooth Conditional Stochastic Optimization

Stochastic Motion Planning as Gaussian Variational Inference: Theory and Algorithms

An Efficient Bayesian Optimization Approach for Automated Optimization of Analog Circuits.

Gaussian Process Policy Optimization

Optimistic Optimization of Gaussian Process Samples

Global Optimization of Gaussian processes

Constrained Iterative LQG for Real-Time Chance-Constrained Gaussian Belief Space Planning

Linear cost and exponentially convergent approximation of Gaussian Matérn processes

Bayesian Optimization through Gaussian Cox Process Models for Spatio-temporal Data

Gaussian agency problems with memory and linear contracts

Function Optimization with Posterior Gaussian Derivative Process

Pseudo-Bayesian Optimization

Deep Gaussian process for multi-objective Bayesian optimization