Abstract:Risk and sparsity requirements often need to be enforced simultaneously in many applications, e.g., in portfolio optimization, assortment planning, and treatment planning. Properly balancing these potentially conflicting requirements entails the formulation of functional constrained optimization with either convex or nonconvex objectives. In this paper, we focus on projection-free methods that can generate a sparse trajectory for solving these challenging functional constrained optimization problems. Specifically, for the convex setting, we propose a Level Conditional Gradient (LCG) method, which leverages a level-set framework to update the approximation of the optimal value and an inner conditional gradient oracle (CGO) for solving mini-max subproblems. We show that the method achieves $\mathcal{O}\big(\frac{1}{\epsilon^2}\log\frac{1}{\epsilon}\big)$ iteration complexity for solving both smooth and nonsmooth cases without dependency on a possibly large size of optimal dual Lagrange multiplier. For the nonconvex setting, we introduce the Level Inexact Proximal Point (IPP-LCG) method and the Direct Nonconvex Conditional Gradient (DNCG) method. The first approach taps into the advantage of LCG by transforming the problem into a series of convex subproblems and exhibits an $\mathcal{O}\big(\frac{1}{\epsilon^3}\log\frac{1}{\epsilon}\big)$ iteration complexity for finding an ($\epsilon,\epsilon$)-KKT point. The DNCG is the first single-loop projection-free method, with iteration complexity bounded by $\mathcal{O}\big(1/\epsilon^4\big)$ for computing a so-called $\epsilon$-Wolfe point. We demonstrate the effectiveness of LCG, IPP-LCG and DNCG by devising formulations and conducting numerical experiments on two risk averse sparse optimization applications: a portfolio selection problem with and without cardinality requirement, and a radiation therapy planning problem in healthcare.

Nonconvex Sparse Logistic Regression Via Proximal Gradient Descent.

Nonconvex Sparse Logistic Regression with Weakly Convex Regularization

A Smoothing Proximal Gradient Algorithm for Nonsmooth Convex Regression with Cardinality Penalty.

A Modified Proximal Gradient Method for a Class of Sparse Optimization Problems

Robust Sparse Recovery Via Non-Convex Optimization

Efficient and robust high-dimensional sparse logistic regression via nonlinear primal-dual hybrid gradient algorithms

On Regularized Sparse Logistic Regression

Feature Screening Strategy for Non-Convex Sparse Logistic Regression with Log Sum Penalty

Smoothing Proximal Gradient Method for General Structured Sparse Learning

Fast Learning with Nonconvex L1-2 Regularization.

Nonconvex Sparse Representation With Slowly Vanishing Gradient Regularizers

A Sparse Logistic Regression Framework by Difference of Convex Functions Programming

Sparse Learning for Large-Scale and High-Dimensional Data: A Randomized Convex-Concave Optimization Approach

Greedy Projected Gradient-Newton Method for Sparse Logistic Regression

On Proximal Gradient Method for the Convex Problems Regularized with the Group Reproducing Kernel Norm

Functional Constrained Optimization for Risk Aversion and Sparsity Control

Fast Learning of Nonconvex `1-2-Regularizer using the Proximal Gradient Algorithm

A Proximal-Gradient Homotopy Method for the Sparse Least-Squares Problem

A Feasible Level Proximal Point Method for Nonconvex Sparse Constrained Optimization

A Proximal-Gradient Method for Constrained Optimization

A Fast Hybrid Algorithm for Large-Scale L1-Regularized Logistic Regression