Abstract:In this paper, we consider an SDP relaxation of the quadratic knapsack problem (QKP). After using the Burer-Monteiro factorization, we get a non-convex optimization problem, whose feasible region is an algebraic variety. Although there might be non-regular points on the algebraic variety, we prove that the algebraic variety is a smooth manifold except for a trivial point for a generic input data. We also analyze the local geometric properties of non-regular points on this algebraic variety. In order to maintain the equivalence between the SDP problem and its non-convex formulation, we derive a new rank condition under which these two problems are equivalent. This new rank condition can be much weaker than the classical rank condition if the coefficient matrix has certain special structures. We also prove that under an appropriate rank condition, any second order stationary point of the non-convex problem is also a global optimal solution without any regularity assumption. This result is distinguished from previous results based on LICQ-like smoothness assumption. With all these theoretical properties, we design an algorithm that equip a manifold optimization method with a strategy to escape from non-optimal non-regular points. Our algorithm can also be used as a heuristic for solving the quadratic knapsack problem. Numerical experiments are conducted to verify the high efficiency and robustness of our algorithm as compared to other SDP solvers and a heuristic method based on dynamic programming. In particular, our algorithm is able to solve the SDP relaxation of a one-million dimensional QKP with a sparse cost matrix very accurately in about 20 minutes on a modest desktop computer.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is the semi - definite programming (SDP) relaxation solution method for the Quadratic Knapsack Problem (QKP). Specifically, the authors propose an effective method to solve the SDP relaxation problem of QKP. By low - rank factorization, the original problem is transformed into a non - convex problem, and the good geometric properties of its feasible region are analyzed.
### Problem Background
The Quadratic Knapsack Problem (QKP) is a classic combinatorial optimization problem, in the following form:
\[
(QKP)\quad\max\left\{x^{\top}Cx:a^{\top}x\leq\tau,x\in\{0, 1\}^{n}\right\}
\]
where \(C\in S^{n}\) is the profit matrix, \(a\in\mathbb{R}^{n}\) is the positive weight vector, and \(\tau > 0\) is the knapsack capacity. When \(n>1\), the problem becomes non - trivial. In addition, it is assumed that the weight vector \(a\) and the capacity \(\tau\) satisfy the conditions: for any \(i\in[n]\), \(0 < a_{i}<\tau\) and \(e^{\top}a>\tau\), where \(e\) is the all - one vector.
### Solution
To solve the SDP relaxation problem of QKP, the authors propose the following improved model:
\[
(SQKE)\quad\min\left\{-\langle C, X\rangle:a^{\top}Xa-\tau a^{\top}x = 0,\text{diag}(X)=x,Y:=\begin{pmatrix}1&x^{\top}\\x&X\end{pmatrix}\in S^{n + 1}_{+}\right\}
\]
Through low - rank factorization, the problem is further transformed into:
\[
(SQKELR)\quad\min\left\{-\langle C, RR^{\top}\rangle:\|a^{\top}R\|^{2}-\tau a^{\top}Re_{1}=0,\text{diag}(RR^{\top})=Re_{1},R\in\mathbb{R}^{n\times r}\right\}
\]
where the first element of \(e_{1}\in\mathbb{R}^{r}\) is 1 and the rest are 0.
### Main Contributions
1. **Modified BM Factorization**: By fixing the first row of the low - rank matrix as \(e_{1}^{\top}\), the constraint \(Y_{11} = 1\) is avoided, and it is proved that there are only a finite number of non - regular points in the feasible region.
2. **Good Geometric Properties**: The geometric properties of \(K_{n,r}\) are analyzed, and it is proved that for the general data \((a,\tau)\) satisfying Assumption 1, \(K_{n,r}\) is smooth everywhere except at the zero point.
3. **New Rank Bound**: A new rank bound is derived, making \((SQKE)\) and \((SQKELR)\) equivalent under this rank bound. For the binary linear knapsack problem, the new rank bound is far superior to the classical rank bound.
4. **No Spurious Local Minima**: It is proved that under appropriate rank conditions, \((SQKELR)\) has no spurious local minima.
5. **Riemannian Optimization Method**: An algorithm that combines the Riemannian optimization method and verification and escape strategies is designed, which can handle non - regular points and is more than 100 times faster than other solvers on high - dimensional problems.
### Conclusion
This paper proposes an efficient and robust solution method by in - depth research on the SDP relaxation problem of QKP, which not only has theoretical breakthroughs but also shows superior performance in practical applications.