Abstract:Risk and sparsity requirements often need to be enforced simultaneously in many applications, e.g., in portfolio optimization, assortment planning, and treatment planning. Properly balancing these potentially conflicting requirements entails the formulation of functional constrained optimization with either convex or nonconvex objectives. In this paper, we focus on projection-free methods that can generate a sparse trajectory for solving these challenging functional constrained optimization problems. Specifically, for the convex setting, we propose a Level Conditional Gradient (LCG) method, which leverages a level-set framework to update the approximation of the optimal value and an inner conditional gradient oracle (CGO) for solving mini-max subproblems. We show that the method achieves $\mathcal{O}\big(\frac{1}{\epsilon^2}\log\frac{1}{\epsilon}\big)$ iteration complexity for solving both smooth and nonsmooth cases without dependency on a possibly large size of optimal dual Lagrange multiplier. For the nonconvex setting, we introduce the Level Inexact Proximal Point (IPP-LCG) method and the Direct Nonconvex Conditional Gradient (DNCG) method. The first approach taps into the advantage of LCG by transforming the problem into a series of convex subproblems and exhibits an $\mathcal{O}\big(\frac{1}{\epsilon^3}\log\frac{1}{\epsilon}\big)$ iteration complexity for finding an ($\epsilon,\epsilon$)-KKT point. The DNCG is the first single-loop projection-free method, with iteration complexity bounded by $\mathcal{O}\big(1/\epsilon^4\big)$ for computing a so-called $\epsilon$-Wolfe point. We demonstrate the effectiveness of LCG, IPP-LCG and DNCG by devising formulations and conducting numerical experiments on two risk averse sparse optimization applications: a portfolio selection problem with and without cardinality requirement, and a radiation therapy planning problem in healthcare.

Minimization Over the Nonconvex Sparsity Constraint Using A Hybrid First-order method

Reduced-Space Iteratively Reweighted Second-Order Methods for Nonconvex Sparse Regularization

Generalized non-convex non-smooth sparse and low rank minimization using proximal average.

An Aggressive Reduction on the Complexity of Optimization for Non-Strongly Convex Objectives

A Feasible Level Proximal Point Method for Nonconvex Sparse Constrained Optimization

An Efficient Algorithm for Non-Convex Sparse Optimization

An Alternating Proximal Splitting Method With Global Convergence For Nonconvex Structured Sparsity Optimization

Two-stage Convex Relaxation Approach to Least Squares Loss Constrained Low-Rank Plus Sparsity Optimization Problems

Semi-Proximal Point Method for Nonsmooth Convex-Concave Minimax Optimization

Functional Constrained Optimization for Risk Aversion and Sparsity Control

Structured Sparsity Optimization with Non-Convex Surrogates of L2,0-Norm: A Unified Algorithmic Framework

Efficient K-Support-norm Regularized Minimization Via Fully Corrective Frank-Wolfe Method.

Improving Sparsity and Scalability in Regularized Nonconvex Truncated-Loss Learning Problems

An inexact regularized proximal Newton method for nonconvex and nonsmooth optimization

Structured Sparsity Optimization with Non-Convex Surrogates of $\ell _{2,0}$ℓ2,0-Norm: A Unified Algorithmic Framework

A Proximal-Gradient Method for Constrained Optimization

An efficient active-set method with applications to sparse approximations and risk minimization

A Single-Loop Stochastic Proximal Quasi-Newton Method for Large-Scale Nonsmooth Convex Optimization

A General Two-Level Subspace Method for Nonlinear Optimization

Stochastic Optimization for Non-convex Inf-Projection Problems