Abstract:Risk and sparsity requirements often need to be enforced simultaneously in many applications, e.g., in portfolio optimization, assortment planning, and treatment planning. Properly balancing these potentially conflicting requirements entails the formulation of functional constrained optimization with either convex or nonconvex objectives. In this paper, we focus on projection-free methods that can generate a sparse trajectory for solving these challenging functional constrained optimization problems. Specifically, for the convex setting, we propose a Level Conditional Gradient (LCG) method, which leverages a level-set framework to update the approximation of the optimal value and an inner conditional gradient oracle (CGO) for solving mini-max subproblems. We show that the method achieves $\mathcal{O}\big(\frac{1}{\epsilon^2}\log\frac{1}{\epsilon}\big)$ iteration complexity for solving both smooth and nonsmooth cases without dependency on a possibly large size of optimal dual Lagrange multiplier. For the nonconvex setting, we introduce the Level Inexact Proximal Point (IPP-LCG) method and the Direct Nonconvex Conditional Gradient (DNCG) method. The first approach taps into the advantage of LCG by transforming the problem into a series of convex subproblems and exhibits an $\mathcal{O}\big(\frac{1}{\epsilon^3}\log\frac{1}{\epsilon}\big)$ iteration complexity for finding an ($\epsilon,\epsilon$)-KKT point. The DNCG is the first single-loop projection-free method, with iteration complexity bounded by $\mathcal{O}\big(1/\epsilon^4\big)$ for computing a so-called $\epsilon$-Wolfe point. We demonstrate the effectiveness of LCG, IPP-LCG and DNCG by devising formulations and conducting numerical experiments on two risk averse sparse optimization applications: a portfolio selection problem with and without cardinality requirement, and a radiation therapy planning problem in healthcare.

Nonconvex Sparse Logistic Regression Via Proximal Gradient Descent.

Nonconvex Sparse Logistic Regression with Weakly Convex Regularization

Convergence of Projected Subgradient Method with Sparse or Low-Rank Constraints

Robust Sparse Recovery Via Non-Convex Optimization

Efficient and robust high-dimensional sparse logistic regression via nonlinear primal-dual hybrid gradient algorithms

On Regularized Sparse Logistic Regression

Smoothing Proximal Gradient Method for General Structured Sparse Learning

Fast Learning with Nonconvex L1-2 Regularization.

Nonconvex Sparse Representation With Slowly Vanishing Gradient Regularizers

Sparse Learning for Large-Scale and High-Dimensional Data: A Randomized Convex-Concave Optimization Approach

Functional Constrained Optimization for Risk Aversion and Sparsity Control

Fast Learning of Nonconvex `1-2-Regularizer using the Proximal Gradient Algorithm

A Proximal-Gradient Homotopy Method for the Sparse Least-Squares Problem

A Feasible Level Proximal Point Method for Nonconvex Sparse Constrained Optimization

A Proximal-Gradient Method for Constrained Optimization

A Non-convex Approach for Sparse Recovery with Convergence Guarantee

Regularized online exponentially concave optimization

A proximal-gradient method for problems with overlapping group-sparse regularization: support identification complexity

From Least Squares to Sparse: A Non-Convex Approach with Guarantee

Learning Model-Based Sparsity via Projected Gradient Descent

Fast Sparse Recovery Via Non-Convex Optimization