Abstract:In the seminal book Méchanique analitique, Lagrange, 1788, the notion of a Lagrange multiplier was first introduced in order to study a smooth minimization problem subject to equality constraints. The idea is that, under some regularity assumption, at a solution of the problem, one may associate a new variable (Lagrange multiplier) to each constraint such that an equilibrium equation is satisfied. This concept turned out to be central in studying more general constrained optimization problems and it has lead to the rapid development of nonlinear programming as a field of mathematics since the works of Karush and Kuhn-Tucker, who considered equality and inequality constraints. The usual proofs for the existence of Lagrange multipliers are somewhat cumbersome, relying on the implicit function theorem or duality theory. In the first section of this note we present an elementary proof of existence of Lagrange multipliers in the simplest context, which is easily accessible to a wide variety of readers. In addition, this proof is readily extended to the much more general context of conic constraints, which we present in the second section together with the background properties needed on the projection onto a closed and convex cone.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is the proof of the existence of Lagrange multipliers. Specifically, the paper provides a concise proof method for verifying the existence of Lagrange multipliers in optimization problems with equality constraints. In addition, the author also extends this proof method to more general cone - constrained optimization problems.
### Existence of Lagrange Multipliers under Equality Constraints
For an optimization problem with equality constraints:
\[ \text{Minimize } f(x), \]
\[ \text{subject to } h_1(x) = 0, \ldots, h_m(x) = 0, \]
where \( f, h_1, \ldots, h_m : \mathbb{R}^n \to \mathbb{R} \) are continuously differentiable functions. If \( x \) is a local solution to this problem and satisfies certain regularity conditions (for example, the gradients of the constraints are linearly independent at \( x \)), then there exist Lagrange multipliers \( \lambda_1, \ldots, \lambda_m \in \mathbb{R} \) such that:
\[ \nabla f(x) + \sum_{i = 1}^m \lambda_i \nabla h_i(x)=0. \]
### Existence of Lagrange Multipliers under Cone Constraints
For an optimization problem with cone constraints:
\[ \text{Minimize } f(x), \]
\[ \text{subject to } h(x) \in K, \]
where \( f : X \to \mathbb{R} \) and \( h : X \to Y \) are continuously differentiable functions, \( X \) and \( Y \) are real finite - dimensional vector spaces, and \( K \subseteq Y \) is a closed convex cone. If \( x \) is a local solution to this problem and satisfies certain regularity conditions (for example, the Robinson condition), then there exists a Lagrange multiplier \( \lambda \in Y \) such that:
\[ \nabla f(x)+D h(x)^* \lambda = 0, \]
\[ \langle h(x), \lambda \rangle = 0, \]
\[ \lambda \in K^{\circ}, \]
where \( D h(x) \) is the derivative of \( h \) at \( x \), \( D h(x)^* \) is its adjoint operator, and \( K^{\circ} \) is the polar cone of \( K \).
### Main Contributions
1. **Simplified Proof**: The paper provides a simple and easy - to - understand proof method applicable to the existence of Lagrange multipliers under equality constraints.
2. **Extension to Cone Constraints**: Further extends this proof method to more general cone - constrained optimization problems, demonstrating the existence of Lagrange multipliers in more extensive cases.
3. **Regularity Conditions**: Discusses different regularity conditions, including linear independence and weaker conditions such as cone - linear independence and the Robinson condition.
Through these contributions, the paper not only simplifies the proof of classical results but also provides a theoretical basis for more complex optimization problems.