The Exact Projective Penalty Method for Constrained Optimization

Vladimir Norkin

2023-12-05

Abstract:A new exact projective penalty method is proposed for the equivalent reduction of constrained optimization problems to nonsmooth unconstrained ones. In the method, the original objective function is extended to infeasible points by summing its value at the projection of an infeasible point on the feasible set with the distance to the projection. Besides Euclidean projections, also a pointed projection in the direction of some fixed internal feasible point can be used. The equivalence means that local and global minimums of the problems coincide. Nonconvex sets with multivalued Euclidean projections are admitted, and the objective function may be lower semicontinuous. The particular case of convex problems is included. The obtained unconstrained or box-constrained problem is solved by a version of the branch and bound method combined with local optimization. In principle, any local optimizer can be used within the branch and bound scheme but in numerical experiments sequential quadratic programming method was successfully used. So the proposed exact penalty method does not assume the existence of the objective function outside the allowable area and does not require the selection of the penalty coefficient.

Optimization and Control

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to effectively convert a constrained optimization problem into an unconstrained optimization problem while keeping the local and global minima of the original problem unchanged. Specifically, the paper proposes a new Exact Projective Penalty Method. This method achieves this conversion by extending the original objective function at infeasible points. The core of the method lies in projecting the infeasible points onto the feasible set and adding the value of the objective function at the projection point to the distance to the projection point, thereby constructing a new unconstrained optimization problem. This method is applicable to non - convex feasible sets and possibly discontinuous objective functions, and does not require the selection of a penalty coefficient.

### Main Contributions
1. **Extension to Non - convex Problems**: The paper extends the Exact Projective Penalty Method from convex problems to non - convex problems, that is, dealing with problems with non - convex constraint sets and lower semi - continuous objective functions.
2. **Non - Euclidean Projection**: Proposes a variant using non - Euclidean projection operations, which may be more computationally efficient than Euclidean projection, especially for certain non - convex problems.
3. **Application to Stochastic Programming Problems**: Applies this method to constrained stochastic programming problems and demonstrates its application effectiveness in practical problems.
4. **Connection with Kirszbraun Theorem**: Establishes the relationship between the proposed Exact Projective Penalty function and the classical Kirszbraun theorem, and proves the effectiveness of this method in maintaining the Lipschitz property.

### Method Overview
- **Basic Idea**: For a constrained optimization problem:
\[
f(x)\to\min\quad x\in C\subseteq\mathbb{R}^n,
\]
where \(f(x)\) is a lower semi - continuous function defined on the closed set \(C\subseteq\mathbb{R}^n\). The paper proposes to convert this problem into the following unconstrained optimization problem:
\[
F(x):=\min_{y\in\pi_C(x)}f(y)+Md_C(x)\to\min_{x\in\mathbb{R}^n},\quad M > 0,
\]
where \(\pi_C(x)\) is the projection of point \(x\) on set \(C\), and \(d_C(x)\) is the distance from point \(x\) to set \(C\).

- **Non - Euclidean Projection**: If a certain feasible point \(x_0\in C\) is known, the projection point \(p_C(x)\) can be found by solving a one - dimensional nonlinear equation, thereby constructing the exact penalty function:
\[
F(x):=f(p_C(x))+Mr_C(x),
\]
where \(r_C(x)=\|x - p_C(x)\|\).

### Theoretical Results
- **Equivalence**: The paper proves that the original constrained optimization problem and the newly constructed unconstrained optimization problem are equivalent, that is, their local and global minima and their corresponding function values are the same.
- **Lipschitz Property**: If the objective function \(f\) is Lipschitz continuous on the closed convex set \(C\), then the constructed exact penalty function \(F(x)\) is also Lipschitz continuous.

### Application Examples
- **Linear Constraint Problems**: The paper demonstrates the effectiveness of the method through specific linear constraint problems.
- **Block Optimization Problems**: Discusses how to convert block optimization problems into unconstrained optimization problems.

### Conclusion
The new Exact Projective Penalty Method proposed in the paper provides an effective method for converting complex constrained optimization problems into unconstrained optimization problems while maintaining the properties of the solutions of the original problems. This method performs well in dealing with non - convex problems and stochastic programming problems and has broad application prospects.