Abstract:This work addresses data-driven inverse optimization (IO), where the goal is to estimate unknown parameters in an optimization model from observed decisions that can be assumed to be optimal or near-optimal solutions to the optimization problem. The IO problem is commonly formulated as a large-scale bilevel program that is notoriously difficult to solve. Deviating from traditional exact solution methods, we propose a derivative-free optimization approach based on Bayesian optimization, which we call BO4IO, to solve general IO problems. We treat the IO loss function as a black box and approximate it with a Gaussian process model. Using the predicted posterior function, an acquisition function is minimized at each iteration to query new candidate solutions and sequentially converge to the optimal parameter estimates. The main advantages of using Bayesian optimization for IO are two-fold: (i) it circumvents the need of complex reformulations of the bilevel program or specialized algorithms and can hence enable computational tractability even when the underlying optimization problem is nonconvex or involves discrete variables, and (ii) it allows approximations of the profile likelihood, which provide uncertainty quantification on the IO parameter estimates. We apply the proposed method to three computational case studies, covering different classes of forward optimization problems ranging from convex nonlinear to nonconvex mixed-integer nonlinear programs. Our extensive computational results demonstrate the efficacy and robustness of BO4IO to accurately estimate unknown model parameters from small and noisy datasets. In addition, the proposed profile likelihood analysis has proven to be effective in providing good approximations of the confidence intervals on the parameter estimates and assessing the identifiability of the unknown parameters.

Optimization-based frequentist confidence intervals for functionals in constrained inverse problems: Resolving the Burrus conjecture

A Surrogate Model Based Nested Optimization Framework for Inverse Problem Considering Interval Uncertainty

On the Frequentist Coverage of Bayes Posteriors in Nonlinear Inverse Problems

Fast and Fair Simultaneous Confidence Bands for Functional Parameters

Overlapping Batch Confidence Intervals on Statistical Functionals Constructed from Time Series: Application to Quantiles, Optimization, and Estimation

Statistical guarantees for Bayesian uncertainty quantification in non-linear inverse problems with Gaussian process priors

High Confidence Level Inference is Almost Free using Parallel Stochastic Optimization

BO4IO: A Bayesian optimization approach to inverse optimization with uncertainty quantification

On the convergence of the Laplace approximation and noise-level-robustness of Laplace-based Monte Carlo methods for Bayesian inverse problems

Data-driven Approximation of Distributionally Robust Chance Constraints Using Bayesian Credible Intervals

Interval uncertainty propagation by a parallel Bayesian global optimization method

Source Condition Double Robust Inference on Functionals of Inverse Problems

Bayesian Approach to Inverse Problems for Functions with A Variable-Index Besov Prior

Convergence Analysis of Surrogate-Based Methods for Bayesian Inverse Problems

Prediction intervals and bands with improved coverage for functional data under noisy discrete observation

A posterior contraction for Bayesian inverse problems in Banach spaces

Better Nonparametric Confidence Intervals Via Robust Bias Correction for Quantile Regression

Conformal Inverse Optimization

Bounds for probabilistic programming with application to a blend planning problem

A unified approach to self-improving property via K-functionals