Abstract:We present a new feasible proximal gradient method for constrained optimization where both the objective and constraint functions are given by the summation of a smooth, possibly nonconvex function and a convex simple function. The algorithm converts the original problem into a sequence of convex subproblems. Formulating those subproblems requires the evaluation of at most one gradient value of the original objective and constraint functions. Either exact or approximate subproblem solutions can be computed efficiently in many cases. An important feature of the algorithm is the constraint level parameter. By carefully increasing this level for each subproblem, we provide a simple solution to overcome the challenge of bounding the Lagrangian multipliers and show that the algorithm follows a strictly feasible solution path till convergence to the stationary point. We develop a simple, proximal gradient descent type analysis, showing that the complexity bound of this new algorithm is comparable to gradient descent for the unconstrained setting, which is new in the literature. Exploiting this new design and analysis technique, we extend our algorithms to some more challenging constrained optimization problems where 1) the objective is a stochastic or finite-sum function, and 2) structured nonsmooth functions replace smooth components of both objective and constraint functions. Complexity results for these problems also seem to be new in the literature. Finally, our method can also be applied to convex function-constrained problems where we show complexities similar to the proximal gradient method.

Iterative algorithms for equilibrium problems based on proximal-like methods

Simple proximal-type algorithms for equilibrium problems

An Efficient Interior-Point Algorithm with New Non-Monotone Line Search Filter Method for Nonlinear Constrained Programming

Proximal Subgradient Algorithm for a Class of Nonconvex Bilevel Equilibrium Problems

On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

A Two-Step Proximal Point Algorithm for Nonconvex Equilibrium Problems with Applications to Fractional Programming

Linearized Proximal Algorithms with Adaptive Stepsizes for Convex Composite Optimization with Applications

Proximal Point Algorithms with Inertial Extrapolation for Quasi-convex Pseudo-monotone Equilibrium Problems

Relaxed-Inertial Proximal Point Algorithms for Nonconvex Equilibrium Problems with Applications

Proximal Point Algorithms on Hadamard Manifolds: Linear Convergence and Finite Termination

A linearly convergent method for solving high-order proximal operator

Convergence rate of inexact proximal point algorithms for operator with holder metric subregularity

A Proximal-Gradient Method for Constrained Optimization

Convergence of inertial prox-penalization and inertial forward–backward algorithms for solving monotone bilevel equilibrium problems

Proximal Projection Method for Stable Linearly Constrained Optimization

Level Constrained First Order Methods for Function Constrained Optimization

An Adapted Proximal Point Algorithm Utilizing the Golden Ratio Technique for Solving Equilibrium Problems in Banach Spaces

An interior proximal gradient method for nonconvex optimization

Convergence results for proximal point algorithm with inertial and correction terms

On the Bredies-Chenchene-Lorenz-Naldi algorithm

A class modified projection algorithms for nonmonotone variational inequalities with continuity