Abstract:This paper focuses on convex constrained optimization problems, where the solution is subject to a convex inequality constraint. In particular, we aim at challenging problems for which both projection into the constrained domain and a linear optimization under the inequality constraint are time-consuming, which render both projected gradient methods and conditional gradient methods (a.k.a. the Frank-Wolfe algorithm) expensive. In this paper, we develop projection reduced optimization algorithms for both smooth and non-smooth optimization with improved convergence rates under a certain regularity condition of the constraint function. We first present a general theory of optimization with only one projection. Its application to smooth optimization with only one projection yields $O(1/\epsilon)$ iteration complexity, which improves over the $O(1/\epsilon^2)$ iteration complexity established before for non-smooth optimization and can be further reduced under strong convexity. Then we introduce a local error bound condition and develop faster algorithms for non-strongly convex optimization at the price of a logarithmic number of projections. In particular, we achieve an iteration complexity of $\widetilde O(1/\epsilon^{2(1-\theta)})$ for non-smooth optimization and $\widetilde O(1/\epsilon^{1-\theta})$ for smooth optimization, where $\theta\in(0,1]$ appearing the local error bound condition characterizes the functional local growth rate around the optimal solutions. Novel applications in solving the constrained $\ell_1$ minimization problem and a positive semi-definite constrained distance metric learning problem demonstrate that the proposed algorithms achieve significant speed-up compared with previous algorithms.

Projected shrinkage algorithm for box-constrained \(\ell _1\) -minimization

An accelerated proximal iterative hard thresholding method for $\ell_0$ minimization

Convergence of Projected Subgradient Method with Sparse or Low-Rank Constraints

A New Proximal Iterative Hard Thresholding Method with Extrapolation for $$\ell _0$$ Minimization

ProPPA: A Fast Algorithm for $\ell_1$ Minimization and Low-Rank Matrix Completion

Solving the L1 regularized least square problem via a box-constrained smooth minimization

Efficient Projected Gradient Methods for L 0 Constrained Optimization

The variant of the iterative shrinkage-thresholding algorithm for minimization of the l 1 over l ∞ norms

An Inertial Shrinking Projection Algorithm for Split Common Fixed Point Problems

Proximal Projection Method for Stable Linearly Constrained Optimization

The Variant of the Iterative Shrinkage-Thresholding Algorithm for Minimization of the ℓ1 over ℓ∞ Norms

Smoothed Proximal Lagrangian Method for Nonlinear Constrained Programs

Inertial hybrid and shrinking projection algorithms for solving variational inequality problems

An interior proximal gradient method for nonconvex optimization

A projected-search interior-point method for nonlinearly constrained optimization

Efficient Projected Gradient Methods for Cardinality Constrained Optimization

Fast projection onto the intersection of simplex and singly linear constraint and its generalized Jacobian

A Richer Theory of Convex Constrained Optimization with Reduced Projections and Improved Rates

Semismooth Newton Algorithm for Efficient Projections Onto ℓ1, ∞-Norm Ball

A Fast Algorithm for Sparse Reconstruction Based on Shrinkage, Subspace Optimization, and Continuation

An improved relaxed inertial projection algorithm for solving the minimum-norm solution of variational inequality and fixed point problems