Abstract:In this paper, we introduce faster accelerated primal-dual algorithms for minimizing a convex function subject to strongly convex function constraints. Prior to our work, the best complexity bound was $\mathcal{O}(1/{\varepsilon})$, regardless of the strong convexity of the constraint function. It is unclear whether the strong convexity assumption can enable even better convergence results. To address this issue, we have developed novel techniques to progressively estimate the strong convexity of the Lagrangian function. Our approach, for the first time, effectively leverages the constraint strong convexity, obtaining an improved complexity of $\mathcal{O}(1/\sqrt{\varepsilon})$. This rate matches the complexity lower bound for strongly-convex-concave saddle point optimization and is therefore order-optimal. We show the superior performance of our methods in sparsity-inducing constrained optimization, notably Google's personalized PageRank problem. Furthermore, we show that a restarted version of the proposed methods can effectively identify the optimal solution's sparsity pattern within a finite number of steps, a result that appears to have independent significance.

A Fast Conjugate Gradient Algorithm with Active Set Prediction for ℓ1 Optimization.

The convergence of conjugate gradient method with nonmonotone line search

Linearized Proximal Algorithms with Adaptive Stepsizes for Convex Composite Optimization with Applications

Efficient Projected Gradient Methods for L 0 Constrained Optimization

A Conjugate Gradient Method with Global Convergence for Large-Scale Unconstrained Optimization Problems

A conjugate gradient method to solve convex constrained monotone equations with applications in compressive sensing

On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

On the Convergence of an Active-Set Method for ℓ1 Minimization

A Projected Conjugate Gradient Method for Box-Constrained Optimization

A Three-Term Conjugate Gradient Algorithm for Large-Scale Unconstrained Optimization Problems

A Descent Conjugate Gradient Algorithm for Optimization Problems and Its Applications in Image Restoration and Compression Sensing

Two sufficient descent three-term conjugate gradient methods for unconstrained optimization problems with applications in compressive sensing

A class of new three-term descent conjugate gradient algorithms for large-scale unconstrained optimization and applications to image restoration problems

On Partly Smoothness, Activity Identification and Faster Algorithms of L 1 over L 2 Minimization

A Fast Method for Lasso and Logistic Lasso

Faster Accelerated First-order Methods for Convex Optimization with Strongly Convex Function Constraints

Solving Large-Scale Unconstrained Optimization Problems with an Efficient Conjugate Gradient Class

An Improved Spectral Conjugate Gradient Algorithm for Nonconvex Unconstrained Optimization Problems

An Alternating Structure-Adapted Bregman Proximal Gradient Descent Algorithm for Constrained Nonconvex Nonsmooth Optimization Problems and Its Inertial Variant.

A New Hybrid Descent Algorithm for Large-Scale Nonconvex Optimization and Application to Some Image Restoration Problems

Several Accelerated Subspace Minimization Conjugate Gradient Methods Based on Regularization Model and Convergence Rate Analysis for Nonconvex Problems.