Abstract:We consider convex optimization with non-smooth objective function and log-concave sampling with non-smooth potential (negative log density). In particular, we study two specific settings where the convex objective/potential function is either semi-smooth or in composite form as the finite sum of semi-smooth components. To overcome the challenges caused by non-smoothness, our algorithms employ two powerful proximal frameworks in optimization and sampling: the proximal point framework for optimization and the alternating sampling framework (ASF) that uses Gibbs sampling on an augmented distribution. A key component of both optimization and sampling algorithms is the efficient implementation of the proximal map by the regularized cutting-plane method. We establish the iteration-complexity of the proximal map in both semi-smooth and composite settings. We further propose an adaptive proximal bundle method for non-smooth optimization. The proposed method is universal since it does not need any problem parameters as input. Additionally, we develop a proximal sampling oracle that resembles the proximal map in optimization and establish its complexity using a novel technique (a modified Gaussian integral). Finally, we combine this proximal sampling oracle and ASF to obtain a Markov chain Monte Carlo method with non-asymptotic complexity bounds for sampling in semi-smooth and composite settings.

Various Notions of Nonexpansiveness Coincide for Proximal Mappings of Functions

A proximal average for prox-bounded functions

On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

An extension of the proximal point algorithm beyond convexity

Convergence analysis of inexact proximal point algorithms on Hadamard manifolds

Proximal Point Algorithms on Hadamard Manifolds: Linear Convergence and Finite Termination

Proximal determination of convex functions

Convergence rate of inexact proximal point algorithms for operator with holder metric subregularity

Linearized Proximal Algorithms with Adaptive Stepsizes for Convex Composite Optimization with Applications

Level proximal subdifferential, variational convexity, and pointwise Lipschitz smoothness

Proximal Algorithms for a class of abstract convex functions

Finite Termination of Inexact Proximal Point Algorithms in Hilbert Spaces

N ov 2 01 6 A Proximal Stochastic Quasi-Newton Algorithm

Monotone vector fields and the proximal point algorithm on Hadamard manifolds

Proximal Oracles for Optimization and Sampling

Error bounds, PL condition, and quadratic growth for weakly convex functions, and linear convergences of proximal point methods

A Bregman firmly nonexpansive proximal operator for baryconvex optimization

Local convergence of an inexact proximal algorithm for weakly convex functions

Every proximal mapping is a resolvent of level proximal subdifferential

The Bregman proximal average

Variational Analysis of Proximal Compositions and Integral Proximal Mixtures