Abstract:We establish strong duality relations for functional two-step compositional risk-constrained learning problems with multiple nonconvex loss functions and/or learning constraints, regardless of nonconvexity and under a minimal set of technical assumptions. Our results in particular imply zero duality gaps within the class of problems under study, both extending and improving on the state of the art in (risk-neutral) constrained learning. More specifically, we consider risk objectives/constraints which involve real-valued convex and positively homogeneous risk measures admitting dual representations with bounded risk envelopes, generalizing expectations and including popular examples, such as the conditional value-at-risk (CVaR), the mean-absolute deviation (MAD), and more generally all real-valued coherent risk measures on integrable losses as special cases. Our results are based on recent advances in risk-constrained nonconvex programming in infinite dimensions, which rely on a remarkable new application of J. J. Uhl's convexity theorem, which is an extension of A. A. Lyapunov's convexity theorem for general, infinite dimensional Banach spaces. By specializing to the risk-neutral setting, we demonstrate, for the first time, that constrained classification and regression can be treated under a unifying lens, while dispensing certain restrictive assumptions enforced in the current literature, yielding a new state-of-the-art strong duality framework for nonconvex constrained learning.

A Note on Duality in Reverse Convex Optimization

A note on duality in reverse convex optimization

A Duality Theory for Non-convex Problems in the Calculus of Variations

Stable and Total Fenchel Duality for Convex Optimization Problems in Locally Convex Spaces

Optimality and duality analysis for multiobjective interval-valued semi-infinite optimization problem having vanishing constraints

Dualities in Convex Algebraic Geometry

Constraint Qualifications for Optimality Conditions and Total Lagrange Dualities in Convex Infinite Programming

Dual Solutions in Convex Stochastic Optimization

Stable and Total Fenchel Duality for DC Optimization Problems in Locally Convex Spaces.

Duality for a Class of Multiobjective Programing under Generalized V-Type I

Lagrange duality on DC evenly convex optimization problems via a generalized conjugation scheme

Strong Duality Relations in Nonconvex Risk-Constrained Learning

Strong Duality in Risk-Constrained Nonconvex Functional Programming

Symmetric norms and reverse inequalities to Davis and Hansen-Pedersen characterizations of operator convexity

A Note on Ordinally Concave Functions

A convex dual variational formulation for non-convex optimization applied to a non-linear model of plates

Optimality conditions and duality for a class of continuous-time linear optimization problems

Duality theory for optimistic bilevel optimization

Optimality and Duality of Semi-Preinvariant Convex Multi-Objective Programming Involving Generalized (F,α,ρ,d)−I−Type Invex Functions

Lossless Convexification and Duality

Optimality and Duality for Robust Optimization Problems Involving Intersection of Closed Sets