Abstract:Risk and sparsity requirements often need to be enforced simultaneously in many applications, e.g., in portfolio optimization, assortment planning, and treatment planning. Properly balancing these potentially conflicting requirements entails the formulation of functional constrained optimization with either convex or nonconvex objectives. In this paper, we focus on projection-free methods that can generate a sparse trajectory for solving these challenging functional constrained optimization problems. Specifically, for the convex setting, we propose a Level Conditional Gradient (LCG) method, which leverages a level-set framework to update the approximation of the optimal value and an inner conditional gradient oracle (CGO) for solving mini-max subproblems. We show that the method achieves $\mathcal{O}\big(\frac{1}{\epsilon^2}\log\frac{1}{\epsilon}\big)$ iteration complexity for solving both smooth and nonsmooth cases without dependency on a possibly large size of optimal dual Lagrange multiplier. For the nonconvex setting, we introduce the Level Inexact Proximal Point (IPP-LCG) method and the Direct Nonconvex Conditional Gradient (DNCG) method. The first approach taps into the advantage of LCG by transforming the problem into a series of convex subproblems and exhibits an $\mathcal{O}\big(\frac{1}{\epsilon^3}\log\frac{1}{\epsilon}\big)$ iteration complexity for finding an ($\epsilon,\epsilon$)-KKT point. The DNCG is the first single-loop projection-free method, with iteration complexity bounded by $\mathcal{O}\big(1/\epsilon^4\big)$ for computing a so-called $\epsilon$-Wolfe point. We demonstrate the effectiveness of LCG, IPP-LCG and DNCG by devising formulations and conducting numerical experiments on two risk averse sparse optimization applications: a portfolio selection problem with and without cardinality requirement, and a radiation therapy planning problem in healthcare.

A first-order optimization algorithm for statistical learning with hierarchical sparsity structure

Structured Sparsity Optimization with Non-Convex Surrogates of L2,0-Norm: A Unified Algorithmic Framework

An Alternating Proximal Splitting Method With Global Convergence For Nonconvex Structured Sparsity Optimization

Characterizing the SLOPE Trade-off: A Variational Perspective and the Donoho-Tanner Limit

Structured Sparsity Optimization with Non-Convex Surrogates of $\ell _{2,0}$ℓ2,0-Norm: A Unified Algorithmic Framework

Smoothing Proximal Gradient Method for General Structured Sparse Learning

An efficient Hessian based algorithm for solving large-scale sparse group Lasso problems

L1-Regularized Distributed Optimization: A Communication-Efficient Primal-Dual Framework

A proximal-gradient method for problems with overlapping group-sparse regularization: support identification complexity

Minimization Over the Nonconvex Sparsity Constraint Using A Hybrid First-order method

Stochastic Iterative Hard Thresholding for Graph-structured Sparsity Optimization

ADMM Algorithmic Regularization Paths for Sparse Statistical Machine Learning

Proximal methods for sparse optimal scoring and discriminant analysis

Efficient sparse Hessian based algorithms for the clustered lasso problem.

Network Lasso: Clustering and Optimization in Large Graphs

Smoothing composite proximal gradient algorithm for sparse group Lasso problems with nonsmooth loss functions

A Sparse-Group Lasso

Functional Constrained Optimization for Risk Aversion and Sparsity Control

Iterative-Weighted Thresholding Method for Group-Sparsity-Constrained Optimization With Applications

Greedy Sparsity-Constrained Optimization