Abstract:This paper investigates the problem of efficient constrained global optimization of hybrid models that are a composition of a known white-box function and an expensive multi-output black-box function subject to noisy observations, which often arises in real-world science and engineering applications. We propose a novel method, Constrained Upper Quantile Bound (CUQB), to solve such problems that directly exploits the composite structure of the objective and constraint functions that we show leads substantially improved sampling efficiency. CUQB is a conceptually simple, deterministic approach that avoid constraint approximations used by previous methods. Although the CUQB acquisition function is not available in closed form, we propose a novel differentiable sample average approximation that enables it to be efficiently maximized. We further derive bounds on the cumulative regret and constraint violation under a non-parametric Bayesian representation of the black-box function. Since these bounds depend sublinearly on the number of iterations under some regularity assumptions, we establis bounds on the convergence rate to the optimal solution of the original constrained problem. In contrast to most existing methods, CUQB further incorporates a simple infeasibility detection scheme, which we prove triggers in a finite number of iterations when the original problem is infeasible (with high probability given the Bayesian model). Numerical experiments on several test problems, including environmental model calibration and real-time optimization of a reactor system, show that CUQB significantly outperforms traditional Bayesian optimization in both constrained and unconstrained cases. Furthermore, compared to other state-of-the-art methods that exploit composite structure, CUQB achieves competitive empirical performance while also providing substantially improved theoretical guarantees.

Kernelized Normalizing Constant Estimation: Bridging Bayesian Quadrature and Bayesian Optimization

On Average-Case Error Bounds for Kernel-Based Bayesian Quadrature

Error Bounds for a Kernel-Based Constrained Optimal Smoothing Approximation

Conditional Bayesian Quadrature

Bayesian Kernelized Tensor Factorization as Surrogate for Bayesian Optimization

On the Sampling Problem for Kernel Quadrature

No-Regret Constrained Bayesian Optimization of Noisy and Expensive Hybrid Models using Differentiable Quantile Function Approximations

Almost-sure quasi-optimal approximation in reproducing kernel Hilbert spaces

Efficient Numerical Integration in Reproducing Kernel Hilbert Spaces via Leverage Scores Sampling

Deep Kernel Learning-Based Bayesian Optimization with Adaptive Kernel Functions

Kernel Bayesian Inference with Posterior Regularization

Achieving High Convergence Rates by Quasi-Monte Carlo and Importance Sampling for Unbounded Integrands

Exploiting gradients and Hessians in Bayesian optimization and Bayesian quadrature

PACSBO: Probably approximately correct safe Bayesian optimization

Statistical Optimality of Divide and Conquer Kernel-based Functional Linear Regression

Learning Analysis of Kernel Ridgeless Regression with Asymmetric Kernel Learning

Quantized Kernel Least Mean Square Algorithm

Beyond Linearization: On Quadratic and Higher-Order Approximation of Wide Neural Networks

Source Condition Double Robust Inference on Functionals of Inverse Problems

Necessary Optimality Conditions and Exact Penalization for Non-Lipschitz Nonlinear Programs

Analysis of Kernel Mirror Prox for Measure Optimization