Abstract:This paper investigates the problem of efficient constrained global optimization of hybrid models that are a composition of a known white-box function and an expensive multi-output black-box function subject to noisy observations, which often arises in real-world science and engineering applications. We propose a novel method, Constrained Upper Quantile Bound (CUQB), to solve such problems that directly exploits the composite structure of the objective and constraint functions that we show leads substantially improved sampling efficiency. CUQB is a conceptually simple, deterministic approach that avoid constraint approximations used by previous methods. Although the CUQB acquisition function is not available in closed form, we propose a novel differentiable sample average approximation that enables it to be efficiently maximized. We further derive bounds on the cumulative regret and constraint violation under a non-parametric Bayesian representation of the black-box function. Since these bounds depend sublinearly on the number of iterations under some regularity assumptions, we establis bounds on the convergence rate to the optimal solution of the original constrained problem. In contrast to most existing methods, CUQB further incorporates a simple infeasibility detection scheme, which we prove triggers in a finite number of iterations when the original problem is infeasible (with high probability given the Bayesian model). Numerical experiments on several test problems, including environmental model calibration and real-time optimization of a reactor system, show that CUQB significantly outperforms traditional Bayesian optimization in both constrained and unconstrained cases. Furthermore, compared to other state-of-the-art methods that exploit composite structure, CUQB achieves competitive empirical performance while also providing substantially improved theoretical guarantees.

Hybrid Parameter Search and Dynamic Model Selection for Mixed-Variable Bayesian Optimization

Hybrid Parameter Search and Dynamic Model Selection for Mixed-Variable Bayesian Optimization

Monte Carlo Tree Search based Variable Selection for High Dimensional Bayesian Optimization

Bayesian Optimization For Multi-Objective Mixed-Variable Problems

A Hybrid Monte Carlo Architecture for Parameter Optimization

High-dimensional Bayesian optimization using low-dimensional feature spaces

Simulation Based Bayesian Optimization

Predictive Modeling through Hyper-Bayesian Optimization

A Bayesian Optimization Framework for Finding Local Optima in Expensive Multimodal Functions

A Bayesian Optimization Framework for Finding Local Optima in Expensive Multi-Modal Functions

Modulating Surrogates for Bayesian Optimization

High-Dimensional Bayesian Optimization via Tree-Structured Additive Models

CatCMA : Stochastic Optimization for Mixed-Category Problems

Bayesian Kernelized Tensor Factorization as Surrogate for Bayesian Optimization

Efficient Hybrid Bayesian Optimization Algorithm with Adaptive Expected Improvement Acquisition Function

Bayesian optimization with adaptive surrogate models for automated experimental design

A comparison of mixed-variables Bayesian optimization approaches

Bayesian Optimization over Discrete and Mixed Spaces via Probabilistic Reparameterization

Hybrid Reinforcement Learning Framework for Mixed-Variable Problems

No-Regret Constrained Bayesian Optimization of Noisy and Expensive Hybrid Models using Differentiable Quantile Function Approximations

Collaborative and Adaptive Bayesian Optimization for bounding variances and probabilities under hybrid uncertainties