Abstract:This paper investigates the problem of efficient constrained global optimization of hybrid models that are a composition of a known white-box function and an expensive multi-output black-box function subject to noisy observations, which often arises in real-world science and engineering applications. We propose a novel method, Constrained Upper Quantile Bound (CUQB), to solve such problems that directly exploits the composite structure of the objective and constraint functions that we show leads substantially improved sampling efficiency. CUQB is a conceptually simple, deterministic approach that avoid constraint approximations used by previous methods. Although the CUQB acquisition function is not available in closed form, we propose a novel differentiable sample average approximation that enables it to be efficiently maximized. We further derive bounds on the cumulative regret and constraint violation under a non-parametric Bayesian representation of the black-box function. Since these bounds depend sublinearly on the number of iterations under some regularity assumptions, we establis bounds on the convergence rate to the optimal solution of the original constrained problem. In contrast to most existing methods, CUQB further incorporates a simple infeasibility detection scheme, which we prove triggers in a finite number of iterations when the original problem is infeasible (with high probability given the Bayesian model). Numerical experiments on several test problems, including environmental model calibration and real-time optimization of a reactor system, show that CUQB significantly outperforms traditional Bayesian optimization in both constrained and unconstrained cases. Furthermore, compared to other state-of-the-art methods that exploit composite structure, CUQB achieves competitive empirical performance while also providing substantially improved theoretical guarantees.

Bayesian Optimization via Exact Penalty

A Novel Penalty Approach for Nonlinear Dynamic Optimization Problems with Inequality Path Constraints

A nonsmooth exact penalty method for equality-constrained optimization: complexity and implementation

Pseudo-Bayesian Optimization

An Exact Penalty Approach for Optimization with Nonnegative Orthogonality Constraints

An Improved Multi-Population Genetic Algorithm For Constrained Nonlinear Optimization

Constraint Handling in Bayesian Optimization -- A Comparative Study of Support Vector Machine, Augmented Lagrangian and Expected Feasible Improvement

A class of smooth exact penalty function methods for optimization problems with orthogonality constraints

Penalty Methods for a Class of Non-Lipschitz Optimization Problems

Bayesian Optimization over Discrete and Mixed Spaces via Probabilistic Reparameterization

No-Regret Constrained Bayesian Optimization of Noisy and Expensive Hybrid Models using Differentiable Quantile Function Approximations

A Bayesian Discrete Optimization Algorithm for Permutation Based Combinatorial Problems

The Exact Projective Penalty Method for Constrained Optimization

A penalty barrier framework for nonconvex constrained optimization

Bayesian Optimization Using Pseudo-Points

Penalty method for the sparse portfolio optimization problem

Bayesian Optimization with Exponential Convergence

Bayesian Optimisation for Constrained Problems

Bi-objective Bayesian optimization of engineering problems with cheap and expensive cost functions

Constrained Bayesian optimization algorithms for estimating design points in structural reliability analysis