Abstract:We present a new feasible proximal gradient method for constrained optimization where both the objective and constraint functions are given by the summation of a smooth, possibly nonconvex function and a convex simple function. The algorithm converts the original problem into a sequence of convex subproblems. Formulating those subproblems requires the evaluation of at most one gradient value of the original objective and constraint functions. Either exact or approximate subproblem solutions can be computed efficiently in many cases. An important feature of the algorithm is the constraint level parameter. By carefully increasing this level for each subproblem, we provide a simple solution to overcome the challenge of bounding the Lagrangian multipliers and show that the algorithm follows a strictly feasible solution path till convergence to the stationary point. We develop a simple, proximal gradient descent type analysis, showing that the complexity bound of this new algorithm is comparable to gradient descent for the unconstrained setting, which is new in the literature. Exploiting this new design and analysis technique, we extend our algorithms to some more challenging constrained optimization problems where 1) the objective is a stochastic or finite-sum function, and 2) structured nonsmooth functions replace smooth components of both objective and constraint functions. Complexity results for these problems also seem to be new in the literature. Finally, our method can also be applied to convex function-constrained problems where we show complexities similar to the proximal gradient method.

Linearly Convergent First-Order Algorithms for Semi-definite Programming

A GLOBALLY AND SUPERLIEARLY CONVERGENT FEASIBLE SQP ALGORITHM FOR DEGENERATE CONSTRAINED OPTIMIZATION

Linear convergence of first order methods for non-strongly convex optimization

Linear Convergence of Subgradient Algorithm for Convex Feasibility on Riemannian Manifolds

Superlinear Convergence of an Interior Point Algorithm on Linear Semi-definite Feasibility Problems

Variational Analysis Perspective on Linear Convergence of Some First Order Methods for Nonsmooth Convex Optimization Problems

Quasi-Slater And Farkas-Minkowski Qualifications For Semi-Infinite Programming With Applications

First-Order Methods for Constrained Convex Programming Based on Linearized Augmented Lagrangian Function

On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

Strong Variational Sufficiency for Nonlinear Semidefinite Programming and its Implications

First-Order Methods for Nonsmooth Nonconvex Functional Constrained Optimization with or without Slater Points

Semi-Smooth Second-order Type Methods for Composite Convex Programs

Optimization on linear matrix inequalities for polynomial systems control

First-Order Methods for Linearly Constrained Bilevel Optimization

Level Constrained First Order Methods for Function Constrained Optimization

Structured Semidefinite Representation of Some Convex Sets

Instance-specific linear relaxations of semidefinite optimization problems

Automated Lyapunov Analysis of Primal-Dual Optimization Algorithms: An Interpolation Approach

A feasible adaptive refinement algorithm for linear semi-infinite optimization

Calibrating Least Squares Semidefinite Programming with Equality and Inequality Constraints

Solving Constrained Variational Inequalities via a First-order Interior Point-based Method