Abstract:Risk and sparsity requirements often need to be enforced simultaneously in many applications, e.g., in portfolio optimization, assortment planning, and treatment planning. Properly balancing these potentially conflicting requirements entails the formulation of functional constrained optimization with either convex or nonconvex objectives. In this paper, we focus on projection-free methods that can generate a sparse trajectory for solving these challenging functional constrained optimization problems. Specifically, for the convex setting, we propose a Level Conditional Gradient (LCG) method, which leverages a level-set framework to update the approximation of the optimal value and an inner conditional gradient oracle (CGO) for solving mini-max subproblems. We show that the method achieves $\mathcal{O}\big(\frac{1}{\epsilon^2}\log\frac{1}{\epsilon}\big)$ iteration complexity for solving both smooth and nonsmooth cases without dependency on a possibly large size of optimal dual Lagrange multiplier. For the nonconvex setting, we introduce the Level Inexact Proximal Point (IPP-LCG) method and the Direct Nonconvex Conditional Gradient (DNCG) method. The first approach taps into the advantage of LCG by transforming the problem into a series of convex subproblems and exhibits an $\mathcal{O}\big(\frac{1}{\epsilon^3}\log\frac{1}{\epsilon}\big)$ iteration complexity for finding an ($\epsilon,\epsilon$)-KKT point. The DNCG is the first single-loop projection-free method, with iteration complexity bounded by $\mathcal{O}\big(1/\epsilon^4\big)$ for computing a so-called $\epsilon$-Wolfe point. We demonstrate the effectiveness of LCG, IPP-LCG and DNCG by devising formulations and conducting numerical experiments on two risk averse sparse optimization applications: a portfolio selection problem with and without cardinality requirement, and a radiation therapy planning problem in healthcare.

KL property of exponent $1/2$ of quadratic functions under nonnegative zero-norm constraints and applications

A GLOBALLY AND SUPERLIEARLY CONVERGENT FEASIBLE SQP ALGORITHM FOR DEGENERATE CONSTRAINED OPTIMIZATION

A Globally and Linearly Convergent PGM for Zero-Norm Regularized Quadratic Optimization with Sphere Constraint

Correction To: Kurdyka–Łojasiewicz Property of Zero-Norm Composite Functions

Kurdyka–Łojasiewicz Exponent via Inf-projection

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

Optimality Conditions and Numerical Algorithms for A Class of Linearly Constrained Minimax Optimization Problems

KL PROPERTY OF EXPONENT 1/2 OF l(2,0) -NORM AND DC REGULARIZED FACTORIZATIONS FOR LOW-RANK MATRIX RECOVERY

Error Bound of Critical Points and KL Property of Exponent 1/2 for Squared F-norm Regularized Factorization

Tighter yet more tractable relaxations and nontrivial instance generation for sparse standard quadratic optimization

On Tractable Convex Relaxations of Standard Quadratic Optimization Problems under Sparsity Constraints

Lower Bound Theory of Nonzero Entries in Solutions of $\Ell_2$-$\Ell_p$ Minimization

Novel Exponential Stability Conditions For A Class Of Interval Projection Neural Networks

Second Order Optimality Conditions and Reformulations for Nonconvex Quadratically Constrained Quadratic Programming Problems

Error bounds, PL condition, and quadratic growth for weakly convex functions, and linear convergences of proximal point methods

On enforcing non-negativity in polynomial approximations in high dimensions

Metric Subregularity of Subdifferential and KL Property of Exponent 1/2.

Functional Constrained Optimization for Risk Aversion and Sparsity Control

Sparse Solutions of a Class of Constrained Optimization Problems.

An Augmented Lagrangian Method for Non-Lipschitz Nonconvex Programming

Extreme Point Pursuit -- Part II: Further Error Bound Analysis and Applications