Abstract:Formulating real-world optimization problems often begins with making predictions from historical data (e.g., an optimizer that aims to recommend fast routes relies upon travel-time predictions). Typically, learning the prediction model used to generate the optimization problem and solving that problem are performed in two separate stages. Recent work has showed how such prediction models can be learned end-to-end by differentiating through the optimization task. Such methods often yield empirical improvements, which are typically attributed to end-to-end making better error tradeoffs than the standard loss function used in a two-stage solution. We refine this explanation and more precisely characterize when end-to-end can improve performance. When prediction targets are stochastic, a two-stage solution must make an a priori choice about which statistics of the target distribution to model-we consider expectations over prediction targets-while an end-to-end solution can make this choice adaptively. We show that the performance gap between a two-stage and end-to-end approach is closely related to the price of correlation concept in stochastic optimization and show the implications of some existing POC results for the predict-then-optimize problem. We then consider a novel and particularly practical setting, where multiple prediction targets are combined to obtain each of the objective function's coefficients. We give explicit constructions where (1) two-stage performs unboundedly worse than end-to-end; and (2) two-stage is optimal. We use simulations to experimentally quantify performance gaps and identify a wide range of real-world applications from the literature whose objective functions rely on multiple prediction targets, suggesting that end-to-end learning could yield significant improvements.

Multi-Task Predict-then-Optimize

Predict-Then-Optimize by Proxy: Learning Joint Models of Prediction and Optimization

Multi-Task Learning as Multi-Objective Optimization

A New Multitask Joint Learning Framework for Expensive Multi-Objective Optimization Problems

Multi-Task Learning with Multi-Task Optimization

The Multi-Task Learning with an Application of Pareto Improvement.

Multi-surrogate Multi-Tasking Optimization of Expensive Problems

Do Current Multi-Task Optimization Methods in Deep Learning Even Help?

Multiobjective Multitasking Optimization Assisted by Multidirectional Prediction Method

Multi-Adaptive Optimization for multi-task learning with deep neural networks

Domain Adaptation Multitask Optimization

Multi-task Combinatorial Optimization: Adaptive Multi-modality Knowledge Transfer by an Explicit Inter-task Distance

Multi-task Optimization Based Co-training for Electricity Consumption Prediction

A Multi-task Learning Approach by Combining Derivative-Free and Gradient Methods.

Combinatorial-hybrid Optimization for Multi-agent Systems under Collaborative Tasks

The Perils of Learning Before Optimizing

Rethinking and Benchmarking Predict-then-Optimize Paradigm for Combinatorial Optimization Problems

Efficient Multi-task Prompt Tuning for Recommendation

Learning Task Relationships in Evolutionary Multitasking for Multiobjective Continuous Optimization

No More Tuning: Prioritized Multi-Task Learning with Lagrangian Differential Multiplier Methods

Parametric-Task MAP-Elites