Abstract:Classical penalty methods solve a sequence of unconstrained problems that put greater and greater stress on meeting the constraints. In the limit as the penalty constant tends to $\infty$, one recovers the constrained solution. In the exact penalty method, squared penalties are replaced by absolute value penalties, and the solution is recovered for a finite value of the penalty constant. In practice, the kinks in the penalty and the unknown magnitude of the penalty constant prevent wide application of the exact penalty method in nonlinear programming. In this article, we examine a strategy of path following consistent with the exact penalty method. Instead of performing optimization at a single penalty constant, we trace the solution as a continuous function of the penalty constant. Thus, path following starts at the unconstrained solution and follows the solution path as the penalty constant increases. In the process, the solution path hits, slides along, and exits from the various constraints. For quadratic programming, the solution path is piecewise linear and takes large jumps from constraint to constraint. For a general convex program, the solution path is piecewise smooth, and path following operates by numerically solving an ordinary differential equation segment by segment. Our diverse applications to a) projection onto a convex set, b) nonnegative least squares, c) quadratically constrained quadratic programming, d) geometric programming, and e) semidefinite programming illustrate the mechanics and potential of path following. The final detour to image denoising demonstrates the relevance of path following to regularized estimation in inverse problems. In regularized estimation, one follows the solution path as the penalty constant decreases from a large value.

A Novel Penalty Approach for Nonlinear Dynamic Optimization Problems with Inequality Path Constraints

An Improved Smoothing Technique-Based Control Vector Parameterization Method for Optimal Control Problems with Inequality Path Constraints

An Effective Simultaneous Approach with Variable Time Nodes for Dynamic Optimization Problems

Fast Engineering Optimization: A Novel Highly Effective Control Parameterization Approach for Industrial Dynamic Processes

A Novel Fast Dynamic Optimization Approach for Complex Multivariable Chemical Process Systems

A Novel Non-Uniform Control Vector Parameterization Approach with Time Grid Refinement for Flight Level Tracking Optimal Control Problems.

An objective penalty function method for nonlinear programming

A Penalty Function Algorithm with Objective Parameters for Nonlinear Mathematical Programming

An Improved Control Vector Iteration Approach for Nonlinear Dynamic Optimization (I) Problems Without Path Constraints

A Smooth Penalty Function Algorithm for Computing Nonlinear Inequality Constraint Optimization Problem

A Partial Exact Penalty Function Approach for Constrained Optimization

Exact Penalty Function Approach to Constrained Optimal Control Problems

Penalty Methods for a Class of Non-Lipschitz Optimization Problems

An Improved Multi-Population Genetic Algorithm For Constrained Nonlinear Optimization

A class of smooth exact penalty function methods for optimization problems with orthogonality constraints

An Exact Penalty Approach for Optimization with Nonnegative Orthogonality Constraints

Control Variable Parameterisation With Penalty Approach For Hypersonic Vehicle Reentry Optimisation

Barrier NLP Methods with Structured Regularization for Optimization of Degenerate Optimization Problems

Steering exact penalty DCA for nonsmooth DC optimization problems with equality and inequality constraints

Structured Regularization In Barrier Nlp For Optimization Models With Dependent Constraints

Path Following in the Exact Penalty Method of Convex Programming