Abstract:Formulating real-world optimization problems often begins with making predictions from historical data (e.g., an optimizer that aims to recommend fast routes relies upon travel-time predictions). Typically, learning the prediction model used to generate the optimization problem and solving that problem are performed in two separate stages. Recent work has showed how such prediction models can be learned end-to-end by differentiating through the optimization task. Such methods often yield empirical improvements, which are typically attributed to end-to-end making better error tradeoffs than the standard loss function used in a two-stage solution. We refine this explanation and more precisely characterize when end-to-end can improve performance. When prediction targets are stochastic, a two-stage solution must make an a priori choice about which statistics of the target distribution to model-we consider expectations over prediction targets-while an end-to-end solution can make this choice adaptively. We show that the performance gap between a two-stage and end-to-end approach is closely related to the price of correlation concept in stochastic optimization and show the implications of some existing POC results for the predict-then-optimize problem. We then consider a novel and particularly practical setting, where multiple prediction targets are combined to obtain each of the objective function's coefficients. We give explicit constructions where (1) two-stage performs unboundedly worse than end-to-end; and (2) two-stage is optimal. We use simulations to experimentally quantify performance gaps and identify a wide range of real-world applications from the literature whose objective functions rely on multiple prediction targets, suggesting that end-to-end learning could yield significant improvements.

Coactive Learning for Locally Optimal Problem Solving

A Local Online Learning Approach for Non-linear Data.

Learning-augmented Algorithms for Online Subset Sum

Active Learning for Cost-Sensitive Classification

A Population-Based Incremental Learning Method for Robust Optimal Solutions

Learning-Augmented Decentralized Online Convex Optimization in Networks

Co-adaptive learning over a countable space

Learning for Robust Combinatorial Optimization: Algorithm and Application

An active learning method for solving competitive multi-agent decision-making and control problems

Robustified Learning for Online Optimization with Memory Costs

Online Passive-Aggressive Total-Error-Rate Minimization

Decentralized Online Learning for Noncooperative Games in Dynamic Environments

An Optimal Algorithm for Online Non-Convex Learning

Surrogate Model Assisted Cooperative Coevolution for Large Scale Optimization

Enhancing Cooperative Coevolution for Large Scale Optimization by Adaptively Constructing Surrogate Models

Decentralized Online Learning for Noncooperative Games in Dynamic Environments

The Perils of Learning Before Optimizing

Enhancing Constraint Acquisition Through Hybrid Learning: An Integration of Passive and Active Learning Strategies

SurCo: Learning Linear Surrogates For Combinatorial Nonlinear Optimization Problems

Pareto-Optimal Algorithms for Learning in Games

Learning to Expand/Contract Pareto Sets in Dynamic Multi-Objective Optimization With a Changing Number of Objectives