Abstract:This paper tackles the problem of minimizing the prediction loss of the optimal solution (PLS) of the MILP with given data, which is one of the inverse optimization problems. While existing methods can approximately solve this problem, their implementation in the high-dimensional case to minimize the PLS is computationally expensive because they are inefficient in reducing the prediction loss of weights (PLW). We propose a fast algorithm for minimizing the PLS of MILP. To demonstrate this property, we attribute the problem of minimizing the PLS to that of minimizing the suboptimality loss (SL), which is convex. If the PLS does not vanish, we can adapt the SL to have the estimated loss (SPO loss) with a positive lower bound, which enables us to evaluate the PLW. Consequently, we prove that the proposed algorithm can effectively reduce the PLW and achieve the minimum value of PLS. Our numerical experiments demonstrated that our algorithm successfully achieved the minimum PLS. Compared to existing methods, our algorithm exhibited a smaller dimensionality effect and minimized the PLS in less than 1/7 the number of iterations. Especially in high dimensions, our algorithm significantly improved the PLS by more than two orders of magnitude compared to existing algorithms.

What problem does this paper attempt to address?

The paper attempts to minimize the prediction loss (PLS) of the optimal solution in the mixed - integer linear programming (MILP) inverse optimization problem given the data. Specifically, although existing methods can approximately solve this problem, in high - dimensional cases, these methods are computationally costly and inefficient, especially performing poorly in reducing the prediction loss of weights (PLW). Therefore, the author proposes a fast algorithm to minimize the PLS of MILP. ### Main contributions of the paper 1. **Proof of equivalence**: The author proves that the vanishing of sub - gradients is equivalent to the vanishing of PLS almost everywhere on $\phi^* \in \Phi$. This proposition enables the minimization problem of PLS to be reduced to the minimization problem of the convex function SL. 2. **Estimation of PLW**: If PLS does not vanish, SL can be adjusted to have an estimated loss with a positive lower bound (SPO loss), thereby evaluating PLW. 3. **Effectiveness of the algorithm**: Based on the above theoretical results, the author proves that the proposed PSGD2 algorithm can effectively reduce PLW and reach the minimum value of PLS. 4. **Experimental verification**: Numerical experiments show that the PSGD2 algorithm successfully minimizes PLS within a finite number of iterations. Compared with existing methods, it has a smaller dimensional effect and requires less than one - seventh of the number of iterations. In particular, when the dimension is 6 or higher, the learning effect of PSGD2 is more than two orders of magnitude higher than that of existing algorithms. ### Mathematical formula representation - The prediction loss (PLS) is defined as: \[ \ell_{\text{pres}}(\phi)=\frac{1}{N} \sum_{n = 1}^N \|a(\phi, s(n))-a(n)\|_2^2 \] - The weight prediction loss (PLW) is defined as: \[ \ell_{\text{prew}}(\phi)=\|\phi-\phi^*\|_2 \] - The sub - optimization loss (SL) is defined as: \[ \ell_{\text{sub}}(\phi)=\frac{1}{N} \sum_{n = 1}^N (\phi^\top a(\phi, s(n))-\phi^\top a(n)) \] ### Conclusion By introducing the PSGD2 algorithm, the author provides an efficient method to minimize PLS in the MILP inverse optimization problem, significantly improving the solution efficiency and accuracy in high - dimensional cases.

A fast algorithm to minimize prediction loss of the optimal solution in inverse optimization problem of MILP

An Off-line Algorithm of Constrained Predictive Control Model Using Linear Matrix Inequalities

Optimal Weighting Distance-Based Similarity for Locally Weighted PLS Modeling

An adaptive strategy to improve the partial least squares model via minimum covariance determinant

Fast Approximate L_infty Minimization: Speeding Up Robust Regression

Learning Effective Loss Functions Efficiently

Maximum Optimality Margin: A Unified Approach for Contextual Linear Programming and Inverse Linear Programming

A Gradient Descent Based Algorithm for ℓ Minimization.

A Population-Based Incremental Learning Method for Robust Optimal Solutions

Fast Algorithm for Constrained Linear Inverse Problems

On the Lower Bound of Minimizing Polyak-Łojasiewicz Functions

Functional Partial Least-Squares: Optimal Rates and Adaptation

Approximation Algorithms for Combinatorial Optimization with Predictions

Learning Surrogate Losses

Solving inverse problems in physics by optimizing a discrete loss: Fast and accurate learning without neural networks

Efficient Algorithms for Sum-of-Minimum Optimization

Learning optimal objective values for MILP

Learning to Optimize for Mixed-Integer Non-linear Programming

Using Optimization to Solve Positive LPs Faster in Parallel

Fast Algorithms for Stackelberg Prediction Game with Least Squares Loss