Abstract:We study the convergence of Langevin-Simulated Annealing type algorithms with multiplicative noise, i.e. for V : R d → R V : \mathbb {R}^d o \mathbb {R} a potential function to minimize, we consider the stochastic differential equation d Y t = − σ σ ⊤ ∇ V ( Y t ) dY_t = - \sigma \sigma ^ op abla V(Y_t) d t + a ( t ) σ ( Y t ) d W t + a ( t ) 2 Υ ( Y t ) d t dt + a(t)\sigma (Y_t)dW_t + a(t)^2\Upsilon (Y_t)dt , where ( W t ) (W_t) is a Brownian motion, where σ : R d → M d ( R ) \sigma : \mathbb {R}^d o \mathcal {M}_d(\mathbb {R}) is an adaptive (multiplicative) noise, where a : R + → R + a : \mathbb {R}^+ o \mathbb {R}^+ is a function decreasing to 0 0 and where Υ \Upsilon is a correction term. This setting can be applied to optimization problems arising in Machine Learning; allowing σ \sigma to depend on the position brings faster convergence in comparison with the classical Langevin equation d Y t = − ∇ V ( Y t ) d t + σ d W t dY_t = - abla V(Y_t)dt + \sigma dW_t . The case where σ \sigma is a constant matrix has been extensively studied; however little attention has been paid to the general case. We prove the convergence for the L 1 L^1 -Wasserstein distance of Y t Y_t and of the associated Euler scheme Y ̄ t \bar {Y}_t to some measure ν ⋆ u ^\star which is supported by argmin ⁡ ( V ) \operatorname {argmin}(V) and give rates of convergence to the instantaneous Gibbs measure ν a ( t ) u _{a(t)} of density ∝ exp ⁡ ( − 2 V ( x ) / a ( t ) 2 ) \propto \exp (-2V(x)/a(t)^2) . To do so, we first consider the case where a a is a piecewise constant function. We find again the classical schedule a ( t ) = A log − 1 / 2 ⁡ ( t ) a(t) = A\log ^{-1/2}(t) . We then prove the convergence for the general case by giving bounds for the Wasserstein distance to the stepwise constant case using ergodicity properties.

Stationary and dynamical properties of finite $N$-unit Langevin models subjected to multiplicative noises

Stochastic Averaging of Strongly Nonlinear Oscillators Under Combined Harmonic and Wide-Band Noise Excitations

Nonstationary Probability Densities of System Response of Strongly Nonlinear Single-Degree-of-freedom System Subject to Modulated White Noise Excitation

The stabilizing role of multiplicative noise in non-confining potentials

A moment approach to non-Gaussian colored noises

Stationary State Distribution and Efficiency Analysis of the Langevin Equation Via Real or Virtual Dynamics.

Time Correlation Functions of Equilibrium and Nonequilibrium Langevin Dynamics: Derivations and Numerics Using Random Numbers

Additive-multiplicative stochastic heat equations, stationary solutions, and Cauchy statistics

Stationary states in two-dimensional systems driven by bivariate Lévy noises

Stochastic modelling of dynamical systems with several attractors

Convergence of Langevin-simulated annealing algorithms with multiplicative noise

Probability distributions for multicomponent systems with multiplicative noise

The Phase Diagram of Neutral Quark Matter

Stationary and dynamical properties of information entropies in nonextensive systems

Numerical analysis and applications of Fokker-Planck equations for stochastic dynamical systems with multiplicative α-stable noises

Large Deviation Principle for Multi-Scale Fully Local Monotone Stochastic Dynamical Systems with Multiplicative Noise

Singularities in d-dimensional Langevin equations with anisotropic multiplicative noise and lack of self-adjointness in the corresponding Schrodinger equation

Second- and third-order properties of multidimensional Langevin equations

Nonlinear stochastic modeling with Langevin regression

Simulation of Stationary Gaussian Noise with Regard to the Langevin Equation with Memory Effect.

Fractional Brownian motion in superharmonic potentials and non-Boltzmann stationary distributions