Abstract:This paper investigates the problem of efficient constrained global optimization of hybrid models that are a composition of a known white-box function and an expensive multi-output black-box function subject to noisy observations, which often arises in real-world science and engineering applications. We propose a novel method, Constrained Upper Quantile Bound (CUQB), to solve such problems that directly exploits the composite structure of the objective and constraint functions that we show leads substantially improved sampling efficiency. CUQB is a conceptually simple, deterministic approach that avoid constraint approximations used by previous methods. Although the CUQB acquisition function is not available in closed form, we propose a novel differentiable sample average approximation that enables it to be efficiently maximized. We further derive bounds on the cumulative regret and constraint violation under a non-parametric Bayesian representation of the black-box function. Since these bounds depend sublinearly on the number of iterations under some regularity assumptions, we establis bounds on the convergence rate to the optimal solution of the original constrained problem. In contrast to most existing methods, CUQB further incorporates a simple infeasibility detection scheme, which we prove triggers in a finite number of iterations when the original problem is infeasible (with high probability given the Bayesian model). Numerical experiments on several test problems, including environmental model calibration and real-time optimization of a reactor system, show that CUQB significantly outperforms traditional Bayesian optimization in both constrained and unconstrained cases. Furthermore, compared to other state-of-the-art methods that exploit composite structure, CUQB achieves competitive empirical performance while also providing substantially improved theoretical guarantees.

An Empirical Quantile Estimation Approach to Nonlinear Optimization Problems with Chance Constraints

An Empirical Quantile Estimation Approach for Chance-Constrained Nonlinear Optimization Problems

A Stochastic Approximation Method for Simulation-Based Quantile Optimization

No-Regret Constrained Bayesian Optimization of Noisy and Expensive Hybrid Models using Differentiable Quantile Function Approximations

Online Quantile Regression

Optimizing Chance-Constrained Submodular Problems with Variable Uncertainties

Convergence Analysis for Mathematical Programs with Distributionally Robust Chance Constraint

Stochastic Approximation for Expectation Objective and Expectation Inequality-Constrained Nonconvex Optimization

Sample Complexity of Chance Constrained Optimization in Dynamic Environment

Chance Constrained Probability Measure Optimization: Problem Formulation, Equivalent Reduction, and Sample-based Approximation

A Quantile Neural Network Framework for Two-stage Stochastic Optimization

Nonconvex and Nonsmooth Approaches for Affine Chance-Constrained Stochastic Programs

Sequential Quadratic Optimization for Stochastic Optimization with Deterministic Nonlinear Inequality and Equality Constraints

Fast Computation of Superquantile-Constrained Optimization Through Implicit Scenario Reduction

Conformal Predictive Programming for Chance Constrained Optimization

Chance-constrained Quasi-Convex Optimization with Application to Data-Driven Switched Systems Control

Generalized ℓ1-Penalized Quantile Regression with Linear Constraints.

A Smoothing Iterative Method for Quantile Regression with Nonconvex \begin{document}$ \ell_p $\end{document} Penalty

Partial functional linear quantile regression

Quantile Optimization via Multiple Timescale Local Search for Black-box Functions

Sparse quantile regression