Abstract:Successful machine learning involves a complete pipeline of data, model, and downstream applications. Instead of treating them separately, there has been a prominent increase of attention within the constrained optimization (CO) and machine learning (ML) communities towards combining prediction and optimization models. The so-called end-to-end (E2E) learning captures the task-based objective for which they will be used for decision making. Although a large variety of E2E algorithms have been presented, it has not been fully investigated how to systematically address uncertainties involved in such models. Most of the existing work considers the uncertainties of ML in the input space and improves robustness through adversarial training. We extend this idea to E2E learning and prove that there is a robustness certification procedure by solving augmented integer programming. Furthermore, we show that neglecting the uncertainty of COs during training causes a new trigger for generalization errors. To include all these components, we propose a unified framework that covers the uncertainties emerging in both the input feature space of the ML models and the COs. The framework is described as a robust optimization problem and is practically solved via end-to-end adversarial training (E2E-AT). Finally, the performance of E2E-AT is evaluated by a real-world end-to-end power system operation problem, including load forecasting and sequential scheduling tasks.

What problem does this paper attempt to address?

The paper primarily aims to address the uncertainty issues in end-to-end (E2E) learning frameworks, particularly how to systematically handle this uncertainty to enhance the robustness of decision-making. The core contribution of the paper is the proposal of a unified framework (E2E-AT) to address the uncertainty issues from different sources in E2E learning. Specifically, the framework focuses on the following aspects: 1. **Uncertainty in E2E Learning**: Traditional E2E learning methods often overlook the uncertainty of the decision optimization model itself when training predictive models. This can lead to inconsistencies between the training process and the actual application process, resulting in generalization errors. 2. **Sources of Uncertainty**: - Uncertainty in the input space: such as noise, outliers, etc., in the dataset. - Uncertainty in the Constrained Optimization (CO) model: including uncertainties in predictive parameters and unpredictable parameters. 3. **Robust Optimization**: To address these uncertainties, the authors propose treating E2E learning as a robust optimization problem and adopt end-to-end adversarial training (E2E-AT) to solve this problem. This method considers not only the uncertainty of input features but also the uncertainty of unpredictable parameters in the CO model. 4. **Certified Robustness**: In addition to using adversarial training to improve the robustness of the model, the authors also discuss how to certify the robustness of the model through mixed-integer linear programming (MILP), especially in certain types of optimization problems (e.g., affine parameterized quadratic programming problems). 5. **Experimental Validation**: The paper conducts experimental validation through a practical power system operation problem, demonstrating the effectiveness of the proposed E2E-AT method. It shows that the method can significantly reduce task-related costs, especially when facing different types of attacks. In summary, this paper aims to address the uncertainty issues in E2E learning through a unified framework, thereby improving the robustness and accuracy of decision-making.

E2E-AT: A Unified Framework for Tackling Uncertainty in Task-aware End-to-end Learning

Multistage Robust Mixed-Integer Optimization under Endogenous Uncertainty

A Unified Framework for Adjustable Robust Optimization with Endogenous Uncertainty

Machine Learning-Based System Optimization and Uncertainty Quantification for Integrated Systems

End-to-End Conformal Calibration for Optimization Under Uncertainty

Error-Driven Uncertainty Aware Training

Uncertainty Set Learning for Adaptive Robust Economic Dispatch

RobustE2E: Exploring the Robustness of End-to-End Autonomous Driving

EEE, Remediating the failure of machine learning models via a network-based optimization patch

Unified Uncertainty Calibration

Uncertainty quantification for deeponets with ensemble kalman inversion

Dual Control of Exploration and Exploitation for Auto-Optimisation Control with Active Learning

TEDL: A Two-stage Evidential Deep Learning Method for Classification Uncertainty Quantification

SDE-Net: Equipping Deep Neural Networks with Uncertainty Estimates

Towards characterizing adversarial defects of deep learning software from the lens of uncertainty

A Comprehensive Survey on Evidential Deep Learning and Its Applications

Multidimensional Uncertainty Quantification for Deep Neural Networks

Efficient Multi-task Uncertainties for Joint Semantic Segmentation and Monocular Depth Estimation

End-to-End Learning for Fair Multiobjective Optimization Under Uncertainty

A Bayesian Deep Learning RUL Framework Integrating Epistemic and Aleatoric Uncertainties

Quantitative Stock Investment by Routing Uncertainty-Aware Trading Experts: A Multi-Task Learning Approach