Abstract:Risk and sparsity requirements often need to be enforced simultaneously in many applications, e.g., in portfolio optimization, assortment planning, and treatment planning. Properly balancing these potentially conflicting requirements entails the formulation of functional constrained optimization with either convex or nonconvex objectives. In this paper, we focus on projection-free methods that can generate a sparse trajectory for solving these challenging functional constrained optimization problems. Specifically, for the convex setting, we propose a Level Conditional Gradient (LCG) method, which leverages a level-set framework to update the approximation of the optimal value and an inner conditional gradient oracle (CGO) for solving mini-max subproblems. We show that the method achieves $\mathcal{O}\big(\frac{1}{\epsilon^2}\log\frac{1}{\epsilon}\big)$ iteration complexity for solving both smooth and nonsmooth cases without dependency on a possibly large size of optimal dual Lagrange multiplier. For the nonconvex setting, we introduce the Level Inexact Proximal Point (IPP-LCG) method and the Direct Nonconvex Conditional Gradient (DNCG) method. The first approach taps into the advantage of LCG by transforming the problem into a series of convex subproblems and exhibits an $\mathcal{O}\big(\frac{1}{\epsilon^3}\log\frac{1}{\epsilon}\big)$ iteration complexity for finding an ($\epsilon,\epsilon$)-KKT point. The DNCG is the first single-loop projection-free method, with iteration complexity bounded by $\mathcal{O}\big(1/\epsilon^4\big)$ for computing a so-called $\epsilon$-Wolfe point. We demonstrate the effectiveness of LCG, IPP-LCG and DNCG by devising formulations and conducting numerical experiments on two risk averse sparse optimization applications: a portfolio selection problem with and without cardinality requirement, and a radiation therapy planning problem in healthcare.

Alternating Projections for Learning with Expectation Constraints

Convergence of Projected Subgradient Method with Sparse or Low-Rank Constraints

Learning with Noisy Labels: Interconnection of Two Expectation-Maximizations

Stochastic Approximation for Expectation Objective and Expectation Inequality-Constrained Nonconvex Optimization

Cross-lingual Pseudo-Projected Expectation Regularization for Weakly Supervised Learning

Adiabatic Persistent Contrastive Divergence Learning

Convergence of Expectation-Maximization Algorithm With Mixed-Integer Optimization

Preference Based Adaptation for Learning Objectives.

Optimal Learning via Moderate Deviations Theory

Offline Learning of Decision Functions in Multiplayer Games with Expectation Constraints

Online Learning under Adversarial Nonlinear Constraints

Functional Constrained Optimization for Risk Aversion and Sparsity Control

Learning Firmly Nonexpansive Operators

Bayesian Optimisation for Constrained Problems

Expectation Maximization Pseudo Labels

Learning via Surrogate PAC-Bayes

Learning Deterministic Surrogates for Robust Convex QCQPs

Near-Optimal Solutions of Constrained Learning Problems

Learning Model Agnostic Explanations via Constraint Programming