Abstract:Although nearly 20 years have passed since its conception, the feasibility pump algorithm remains a widely used heuristic to find feasible primal solutions to mixed-integer linear problems. Many extensions of the initial algorithm have been proposed. Yet, its core algorithm remains centered around two key steps: solving the linear relaxation of the original problem to obtain a solution that respects the constraints, and rounding it to obtain an integer solution. This paper shows that the traditional feasibility pump and many of its follow-ups can be seen as gradient-descent algorithms with specific parameters. A central aspect of this reinterpretation is observing that the traditional algorithm differentiates the solution of the linear relaxation with respect to its cost. This reinterpretation opens many opportunities for improving the performance of the original algorithm. We study how to modify the gradient-update step as well as extending its loss function. We perform extensive experiments on MIPLIB instances and show that these modifications can substantially reduce the number of iterations needed to find a solution.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: how to improve the traditional Feasibility Pump algorithm to solve the Mixed - Integer Linear Optimization Problem (MILP) more effectively. Specifically, the author proposes a new interpretation and extension, that is, regarding the traditional Feasibility Pump as a special form of the gradient - descent algorithm, and improving its performance by introducing a differentiable loss function and update steps.
### Core Contributions of the Paper
1. **Re - interpreting the Feasibility Pump**:
- The author re - interprets the traditional Feasibility Pump algorithm as a gradient - descent algorithm with specific loss functions, gradient estimates, and gradient - update parameters. This perspective reveals the differential treatment of solutions in the traditional algorithm when solving linear relaxations.
2. **Proposing the Differentiable Feasibility Pump**:
- Based on the above re - interpretation, the author proposes a generalized Feasibility Pump algorithm - the Differentiable Feasibility Pump, and shows that this generalized form can cover many existing algorithms.
3. **Introducing a New Loss Function**:
- The author introduces a new differentiable rounding operation, allowing the addition of terms that measure the infeasibility of rounded solutions in the loss function. This provides a new method to solve the "blindness" of the traditional Feasibility Pump.
4. **Experimental Verification**:
- Through extensive experiments on MIPLIB instances, the author shows that the original Feasibility Pump has significant stability with respect to hyper - parameters. However, introducing the feasibility loss and adjusting the hyper - parameters can significantly improve performance. In particular, the number of restart operations required for the differentiable pump using only the feasibility loss is reduced by an order of magnitude, making the algorithm more stable.
### Summary of Mathematical Formulas
- **Distance Function**:
\[
\Delta(x, \lfloor x \rceil) = \sum_{i} |x_i - \lfloor x \rceil_i|
\]
where \(\lfloor x \rceil\) represents element - wise rounding of \(x\).
- **Integer Loss Function**:
\[
f(x) = \sum_{i = 1}^{n} (\min\{x_i, 1 - x_i\})^p
\]
where \(p>0\) is the loss order.
- **Regularized Loss Function**:
\[
L(\theta) = f(\hat{x}(\theta))+\gamma\|\theta\|_2^2
\]
where \(\gamma > 0\) is the regularization weight.
- **Gradient Update**:
\[
\theta\leftarrow\theta-\eta\nabla_\theta L(\theta)
\]
where \(\eta > 0\) is the step size.
- **Feasibility Loss**:
\[
g(\tilde{x})=\frac{1}{m}\sum_{j = 1}^{m}\max(s_j(\tilde{x})-\epsilon, 0)
\]
where \(s_j(\tilde{x})=\frac{b_j - A_j\tilde{x}}{\| [A_j, b_j] \|_2}\), \(\epsilon > 0\) is a small constant.
- **Jacobian Matrix of the Differentiable Rounding Operation**:
\[
J_x\lfloor x \rceil_\epsilon=\frac{1}{\epsilon}\phi\left(\frac{0.5 - x}{\epsilon}\right)
\]
where \(\phi\) is the probability density function of the standard Gaussian distribution.
Through these improvements, the author not only provides a new understanding of the traditional Feasibility Pump but also proposes specific optimization methods, making it perform better when solving large - scale Mixed - Integer Linear Optimization Problems.