Abstract:In this paper, we study linearly constrained optimization problems (LCP). After applying Hadamard parametrization, the feasible set of the parametrized problem (LCPH) becomes an algebraic variety, with conducive geometric properties which we explore in depth. We derive explicit formulas for the tangent cones and second-order tangent sets associated with the parametrized polyhedra. Based on these formulas, we develop a procedure to recover the Lagrangian multipliers associated with the constraints to verify the optimality conditions of the given primal variable without requiring additional constraint qualifications. Moreover, we develop a systematic way to stratify the variety into a disjoint union of finitely many Riemannian manifolds. This leads us to develop a hybrid algorithm combining Riemannian optimization and projected gradient to solve (LCP) with convergence guarantees. Numerical experiments are conducted to verify the effectiveness of our method compared with various state-of-the-art algorithms.
What problem does this paper attempt to address?
### The problems the paper attempts to solve
The paper "Optimization over convex polyhedra via Hadamard parametrizations" aims to solve the linear - constrained optimization problem (LCP) on convex polyhedra. Specifically, by applying the Hadamard parametrization method, the paper transforms the original problem into a new non - linear and non - convex problem (LCPH), thus exploring new optimization methods.
### Problem background
The linear - constrained optimization problem (LCP) usually has the following form:
\[ \min \{ \phi(x) : Ax = b, x \in \mathbb{R}^n_+ \} \]
where \(\phi: \mathbb{R}^n \to \mathbb{R}\), \(A = [A_1; A_2; \ldots; A_m] \in \mathbb{R}^{m \times n}\), \(b \in \mathbb{R}^m\). This problem has wide applications in many fields, such as portfolio optimization and machine learning.
### Existing methods and their limitations
Currently, there are many algorithms for solving the LCP problem, such as the simplex method and the interior - point method. However, these methods mainly rely on the properties of convexity and linearity. For specific types of convex polyhedra, such as Birkhoff polyhedra and transportation matrices, the entropy regularization method can provide a lower per - iteration computational complexity. Nevertheless, these methods still have limitations when dealing with large - scale structured problems.
### Hadamard parametrization method
To overcome the limitations of the above methods, the paper introduces the Hadamard parametrization method. The specific steps are as follows:
1. **Variable transformation**: Transform \(x\) into \(y \circ y\), where \(y \in \mathbb{R}^n\). This makes the non - negative constraint \(x \in \mathbb{R}^n_+\) naturally satisfied.
2. **Transformed optimization problem**: The new optimization problem becomes:
\[ \min \{ f(y) := \phi(y \circ y) : A(y \circ y) = b, y \in \mathbb{R}^n \} \]
### Main contributions
1. **Analysis of geometric properties**: The paper studies in detail the geometric properties of the feasible set \(P_{A,b}\) of the parametrized problem \(LCPH\), including explicit formulas for the tangent cone and the second - order tangent set.
2. **Optimality conditions**: Based on these geometric properties, the paper derives the first - order and second - order optimality conditions of \(LCPH\), and proposes a systematic method to recover Lagrange multipliers and verify the optimality conditions.
3. **Manifold decomposition**: Decompose \(P_{A,b}\) into the disjoint union of a finite number of Riemann manifolds, so that Riemann optimization methods can be used.
4. **Hybrid algorithm**: Combine Riemann optimization and the projected gradient method to design a hybrid algorithm with convergence guarantees.
### Numerical experiments
The paper verifies the effectiveness of the proposed method through numerical experiments and compares it with existing advanced algorithms.
### Conclusion
Through the Hadamard parametrization method, the paper provides a new perspective and an effective method for solving the linear - constrained optimization problem on convex polyhedra. This method can not only handle non - convex problems, but also show good performance in practical applications.