Abstract:Risk and sparsity requirements often need to be enforced simultaneously in many applications, e.g., in portfolio optimization, assortment planning, and treatment planning. Properly balancing these potentially conflicting requirements entails the formulation of functional constrained optimization with either convex or nonconvex objectives. In this paper, we focus on projection-free methods that can generate a sparse trajectory for solving these challenging functional constrained optimization problems. Specifically, for the convex setting, we propose a Level Conditional Gradient (LCG) method, which leverages a level-set framework to update the approximation of the optimal value and an inner conditional gradient oracle (CGO) for solving mini-max subproblems. We show that the method achieves $\mathcal{O}\big(\frac{1}{\epsilon^2}\log\frac{1}{\epsilon}\big)$ iteration complexity for solving both smooth and nonsmooth cases without dependency on a possibly large size of optimal dual Lagrange multiplier. For the nonconvex setting, we introduce the Level Inexact Proximal Point (IPP-LCG) method and the Direct Nonconvex Conditional Gradient (DNCG) method. The first approach taps into the advantage of LCG by transforming the problem into a series of convex subproblems and exhibits an $\mathcal{O}\big(\frac{1}{\epsilon^3}\log\frac{1}{\epsilon}\big)$ iteration complexity for finding an ($\epsilon,\epsilon$)-KKT point. The DNCG is the first single-loop projection-free method, with iteration complexity bounded by $\mathcal{O}\big(1/\epsilon^4\big)$ for computing a so-called $\epsilon$-Wolfe point. We demonstrate the effectiveness of LCG, IPP-LCG and DNCG by devising formulations and conducting numerical experiments on two risk averse sparse optimization applications: a portfolio selection problem with and without cardinality requirement, and a radiation therapy planning problem in healthcare.

Cardinality-Constrained Multi-Objective Optimization: Novel Optimality Conditions and Algorithms

Cardinality-Constrained Multi-objective Optimization: Novel Optimality Conditions and Algorithms

A penalty decomposition approach for multi-objective cardinality-constrained optimization problems

Optimization on Pareto sets: On a theory of multi-objective optimization

A sequential optimality condition for Mathematical Programs with Cardinality Constraints

Scaled-PAKKT sequential optimality condition for multiobjective problems and its application to an Augmented Lagrangian method

Optimality Conditions for Cardinality-Constrained Programs and a SCA Method.

The sparse(st) optimization problem: reformulations, optimality, stationarity, and numerical results

Solution sets of three sparse optimization problems for multivariate regression

Cardinality Constrained Portfolio Optimization via Alternating Direction Method of Multipliers

A limited memory Quasi-Newton approach for multi-objective optimization

Mathematical Programs with Complementarity Constraints and a Non-Lipschitz Objective: Optimality and Approximation.

Cardinality-constrained Programs with Nonnegative Variables and an SCA Method

The Multi-Objective Polynomial Optimization

Functional Constrained Optimization for Risk Aversion and Sparsity Control

Optimality and Complexity for Constrained Optimization Problems with Nonconvex Regularization.

Sequential M-Stationarity Conditions for General Optimization Problems

A Relaxed Optimization Approach for Cardinality-Constrained Portfolio Optimization

An Asymptotic Weak Maximum Principle

First- and second-order optimality conditions of nonsmooth sparsity multiobjective optimization via variational analysis