Abstract:We demonstrate that for strongly log-convex densities whose potentials are discontinuous on manifolds, the ULA algorithm converges with stepsize bias of order $1/2$ in Wasserstein-p distance. Our resulting bound is then of the same order as the convergence of ULA for gradient Lipschitz potential. Additionally, we show that so long as the gradient of the potential obeys a growth bound (therefore imposing no regularity condition), the algorithm has stepsize bias of order $1/4$. We therefore unite two active areas of research: i) the study of numerical methods for SDEs with discontinuous coefficients and ii) the study of the non-asymptotic bias of the ULA algorithm (and variants). In particular this is the first result of the former kind we are aware of on an unbounded time interval.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the convergence properties of the Unadjusted Langevin Algorithm (ULA) under strongly log - convex density functions with discontinuous gradients. Specifically, the authors explore the order of step - size bias of the ULA algorithm in Wasserstein - p distance under these conditions and prove that even when the gradient is discontinuous, the algorithm can still converge at a specific order. In addition, they also study the order of step - size bias of the algorithm when the gradient satisfies the growth condition but has no regularity requirement. ### Core Problems of the Paper 1. **Convergence under Strongly Log - Convex Density Functions** - The authors prove that for a strongly log - convex density function $ U $, when its potential function is discontinuous on some manifolds, the order of step - size bias of the ULA algorithm in Wasserstein - p distance is $ O(\gamma^{1/2}) $. - This result is the same as the convergence of the ULA algorithm under the gradient Lipschitz condition. 2. **The Case Where the Gradient Only Satisfies the Growth Condition** - When the gradient only satisfies the linear growth condition without regularity requirements, the authors prove that the order of step - size bias of the ULA algorithm in Wasserstein - p distance is $ O(\gamma^{1/4}) $. - This shows that even when the gradient is very irregular, the ULA algorithm still has a certain degree of robustness. ### Mathematical Expressions - **Strongly Log - Convex Density Function** \[ \langle \nabla U(x)-\nabla U(y), x - y\rangle\geq\mu\|x - y\|^2,\quad\forall x, y\in\mathbb{R}^d \] where $\mu > 0$ is the strong monotonicity constant. - **Convergence in Wasserstein - p Distance** - For the case of discontinuous gradients: \[ W_p(\pi_\beta, L(x_n))\leq W_p(\xi,\pi_\beta)e^{-\mu\gamma n}+c\gamma^{1/2} \] - For the case where the gradient only satisfies the growth condition: \[ W_p(\pi_\beta, L(x_n))\leq W_p(\xi,\pi_\beta)e^{-\mu\gamma n}+c\gamma^{1/4} \] ### Main Contributions 1. **Unifying Two Research Directions** - This paper combines two main directions in numerical method research: one is the method for dealing with discontinuous coefficients of SDEs, and the other is the study of non - asymptotic bias. - The authors are the first to prove the convergence of the ULA algorithm under discontinuous gradient conditions in an unbounded time interval. 2. **Technical Challenges** - The authors overcome the challenges of the unbounded time interval by introducing an appropriate exponentially weighted difference process and applying some key lemmas (such as the local time formula). 3. **Theoretical Significance** - The results of this paper not only extend the existing theory but also provide a theoretical basis for dealing with more complex and practical optimization and sampling problems. ### Conclusion This paper fills the gap in the performance analysis of the algorithm in the case of discontinuous gradients by conducting an in - depth study of the convergence of the ULA algorithm under strongly log - convex density functions. These results are of great significance for understanding and improving sampling and optimization algorithms based on Langevin dynamics.

Convergence Of The Unadjusted Langevin Algorithm For Discontinuous Gradients

Convergence of Unadjusted Langevin in High Dimensions: Delocalization of Bias

Non-asymptotic convergence bounds for modified tamed unadjusted Langevin algorithm in non-convex setting

Analysis of Primal-Dual Langevin Algorithms

Langevin dynamics based algorithm e-TH$\varepsilon$O POULA for stochastic optimization problems with discontinuous stochastic gradient

Sampling as optimization in the space of measures: The Langevin dynamics as a composite optimization problem

Global Convergence of Langevin Dynamics Based Algorithms for Nonconvex Optimization

Langevin Dynamics: A Unified Perspective on Optimization via Lyapunov Potentials

Proximal Langevin Algorithm: Rapid Convergence Under Isoperimetry

Convergence of Constant Step Stochastic Gradient Descent for Non-Smooth Non-Convex Functions

On Independent Samples Along the Langevin Diffusion and the Unadjusted Langevin Algorithm

Projected Langevin Monte Carlo algorithms in non-convex and super-linear setting

Non-asymptotic convergence analysis of the stochastic gradient Hamiltonian Monte Carlo algorithm with discontinuous stochastic gradient with applications to training of ReLU neural networks

Subgradient Langevin Methods for Sampling from Non-smooth Potentials

On Linear Convergence of Non-Euclidean Gradient Methods Without Strong Convexity and Lipschitz Gradient Continuity.

Convergence of Langevin-simulated annealing algorithms with multiplicative noise

Convergence of kinetic Langevin samplers for non-convex potentials

High Probability Convergence Bounds for Non-convex Stochastic Gradient Descent with Sub-Weibull Noise

Smoothing unadjusted Langevin algorithms for nonsmooth composite potential functions

User-friendly guarantees for the Langevin Monte Carlo with inaccurate gradient

Primal Dual Interpretation of the Proximal Stochastic Gradient Langevin Algorithm