Abstract:We study optimization problems whereby the optimization variable is a probability measure. Since the probability space is not a vector space, many classical and powerful methods for optimization (e.g., gradients) are of little help. Thus, one typically resorts to the abstract machinery of infinite-dimensional analysis or other ad-hoc methodologies, not tailored to the probability space, which however involve projections or rely on convexity-type assumptions. We believe instead that these problems call for a comprehensive methodological framework for calculus in probability spaces. In this work, we combine ideas from optimal transport, variational analysis, and Wasserstein gradient flows to equip the Wasserstein space (i.e., the space of probability measures endowed with the Wasserstein distance) with a variational structure, both by combining and extending existing results and introducing novel tools. Our theoretical analysis culminates in very general necessary optimality conditions for optimality. Notably, our conditions (i) resemble the rationales of Euclidean spaces, such as the Karush-Kuhn-Tucker and Lagrange conditions, (ii) are intuitive, informative, and easy to study, and (iii) yield closed-form solutions or can be used to design computationally attractive algorithms. We believe this framework lays the foundation for new algorithmic and theoretical advancements in the study of optimization problems in probability spaces, which we exemplify with numerous case studies and applications to machine learning, drug discovery, and distributionally robust optimization.

Variational Analysis of Proximal Compositions and Integral Proximal Mixtures

Integral Resolvent and Proximal Mixtures

Resolvent and Proximal Compositions

Variational Inequalities for Set-Valued Vector Fields on Riemannian Manifolds: Convexity of the Solution Set and the Proximal Point Algorithm.

Convergence analysis of inexact proximal point algorithms on Hadamard manifolds

Variational Theory and Algorithms for a Class of Asymptotically Approachable Nonconvex Problems

On second-order variational analysis of variational convexity of prox-regular functions

On a decomposition formula for the proximal operator of the sum of two convex functions

Proximal determination of convex functions

Approximate proximal methods for variational inequalities on Hadamard manifolds

Proximal Point Algorithms on Hadamard Manifolds: Linear Convergence and Finite Termination

On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

Monotone vector fields and the proximal point algorithm on Hadamard manifolds

Signal Recovery with Proximal Comixtures

Convergence Analysis of a Method for Variational Inclusions

A modified proximal contraction principle with applications to variational inequality problems

Monotone Operator Theory in Convex Optimization

Variational Analysis in the Wasserstein Space

Resolvent of the parallel composition and proximity operator of the infimal postcomposition

Various Notions of Nonexpansiveness Coincide for Proximal Mappings of Functions

Feasibility problems via paramonotone operators in a convex setting