Abstract:Risk and sparsity requirements often need to be enforced simultaneously in many applications, e.g., in portfolio optimization, assortment planning, and treatment planning. Properly balancing these potentially conflicting requirements entails the formulation of functional constrained optimization with either convex or nonconvex objectives. In this paper, we focus on projection-free methods that can generate a sparse trajectory for solving these challenging functional constrained optimization problems. Specifically, for the convex setting, we propose a Level Conditional Gradient (LCG) method, which leverages a level-set framework to update the approximation of the optimal value and an inner conditional gradient oracle (CGO) for solving mini-max subproblems. We show that the method achieves $\mathcal{O}\big(\frac{1}{\epsilon^2}\log\frac{1}{\epsilon}\big)$ iteration complexity for solving both smooth and nonsmooth cases without dependency on a possibly large size of optimal dual Lagrange multiplier. For the nonconvex setting, we introduce the Level Inexact Proximal Point (IPP-LCG) method and the Direct Nonconvex Conditional Gradient (DNCG) method. The first approach taps into the advantage of LCG by transforming the problem into a series of convex subproblems and exhibits an $\mathcal{O}\big(\frac{1}{\epsilon^3}\log\frac{1}{\epsilon}\big)$ iteration complexity for finding an ($\epsilon,\epsilon$)-KKT point. The DNCG is the first single-loop projection-free method, with iteration complexity bounded by $\mathcal{O}\big(1/\epsilon^4\big)$ for computing a so-called $\epsilon$-Wolfe point. We demonstrate the effectiveness of LCG, IPP-LCG and DNCG by devising formulations and conducting numerical experiments on two risk averse sparse optimization applications: a portfolio selection problem with and without cardinality requirement, and a radiation therapy planning problem in healthcare.

The Complexity of Large-scale Convex Programming under a Linear Optimization Oracle

Uniformly Optimal and Parameter-free First-order Methods for Convex and Function-constrained Optimization

On Lower Complexity Bounds for Large-Scale Smooth Convex Optimization

On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

Oracle Complexity Separation in Convex Optimization

A Constrained Optimization Approach for Lcp

Decomposable Non-Smooth Convex Optimization with Nearly-Linear Gradient Oracle Complexity

An optimal method for stochastic composite optimization

Generalized Uniformly Optimal Methods for Nonlinear Programming

First-order Methods for Affinely Constrained Composite Non-convex Non-smooth Problems: Lower Complexity Bound and Near-optimal Methods

Optimality and Complexity for Constrained Optimization Problems with Nonconvex Regularization.

Iteration-complexity of first-order augmented Lagrangian methods for convex programming

Oracle Complexity of Single-Loop Switching Subgradient Methods for Non-Smooth Weakly Convex Functional Constrained Optimization

Optimal methods for convex nested stochastic composite optimization

Functional Constrained Optimization for Risk Aversion and Sparsity Control

A Modified Hestenes-Stiefel Conjugate Gradient Algorithm for Large-Scale Optimization

Low-Complexity Iterative Methods for Complex-Variable Matrix Optimization Problems in Frobenius Norm

Bundle-level type methods uniformly optimal for smooth and nonsmooth convex optimization

A General Analysis Framework of Lower Complexity Bounds for Finite-Sum Optimization

On the Oracle Complexity of Higher-Order Smooth Non-Convex Finite-Sum Optimization