Abstract:We propose a flexible gradient-based framework for learning linear programs from optimal decisions. Linear programs are often specified by hand, using prior knowledge of relevant costs and constraints. In some applications, linear programs must instead be learned from observations of optimal decisions. Learning from optimal decisions is a particularly challenging bi-level problem, and much of the related inverse optimization literature is dedicated to special cases. We tackle the general problem, learning all parameters jointly while allowing flexible parametrizations of costs, constraints, and loss functions. We also address challenges specific to learning linear programs, such as empty feasible regions and non-unique optimal decisions. Experiments show that our method successfully learns synthetic linear programs and minimum-cost multi-commodity flow instances for which previous methods are not directly applicable. We also provide a fast batch-mode PyTorch implementation of the homogeneous interior point algorithm, which supports gradients by implicit differentiation or backpropagation.
What problem does this paper attempt to address?
This paper attempts to solve the problem of learning linear programming (LP) from optimal decisions. Specifically, traditional linear programming is usually designed by manually setting costs and constraints, but in this paper, the author proposes a flexible gradient - based method that can automatically learn all parameters of the linear programming model from the observed optimal decisions.
### Main Problem Description
1. **Learning Linear Programming from Optimal Decisions**:
- Linear programming is usually manually specified by known costs and constraints.
- In some application scenarios, linear programming needs to be learned from the observed data of optimal decisions instead of directly specifying costs or constraints.
2. **Challenges of Bi - level Optimization Problems**:
- Learning linear programming from optimal decisions is a challenging bi - level optimization problem.
- The relevant inverse optimization literature mainly focuses on specific special cases, while this paper aims to solve the general problem, that is, learning all parameters simultaneously and allowing flexible parameterization of costs, constraints and loss functions.
3. **Specific Challenges**:
- Situations where the feasible region is empty or the optimal solution is not unique.
- How to ensure that the learned linear programming model can also generalize well under new conditions.
### Solutions
- **Bi - level Optimization Framework**:
- A new bi - level optimization formula is proposed, using the gradient descent method to solve this problem.
- Calculate the gradient through back - propagation or implicit differentiation, thereby effectively learning linear programming instances of different scales.
- **Experimental Verification**:
- Experiments show that this method can successfully learn synthetic linear programming and the minimum - cost multi - commodity flow problem, which are not applicable to previous methods.
- A fast batch - mode PyTorch implementation is provided, which supports calculating gradients through implicit differentiation or back - propagation.
### Formula Representation
- **Linear Programming (LP)**:
\[
\min_x c^T x \quad \text{s.t.} \quad Ax \leq b, \quad Gx = h
\]
- **Parameterized Linear Programming (PLP)**:
\[
\min_x c(u)^T x \quad \text{s.t.} \quad A(u)x \leq b(u), \quad G(u)x = h(u)
\]
- **Forward Optimization Problem (FOP)**:
\[
\min_x c(u,w)^T x \quad \text{s.t.} \quad A(u,w)x \leq b(u,w), \quad G(u,w)x = h(u,w)
\]
### Conclusion
This paper proposes a novel gradient - based learning framework that can learn all parameters of the linear programming model from the observed optimal decisions. This method not only solves the limitations in existing inverse optimization methods but also can work effectively in various complex situations, providing new ideas and tools for the learning of linear programming.