Abstract:Global optimization of a non-convex objective function often appears in large-scale machine learning and artificial intelligence applications. Recently, consensus-based optimization (CBO) methods have been introduced as one of the gradient-free optimization methods. In this paper, we provide a convergence analysis for the first-order CBO method in [J. A. Carrillo, S. Jin, L. Li and Y. Zhu, A consensus-based global optimization method for high dimensional machine learning problems, https://arxiv.org/abs/1909.09249v1 ]. Prior to this work, the convergence study was carried out for CBO methods on corresponding mean-field limit, a Fokker–Planck equation, which does not imply the convergence of the CBO method per se. Based on the consensus estimate directly on the first-order CBO model, we provide a convergence analysis of the first-order CBO method [J. A. Carrillo, S. Jin, L. Li and Y. Zhu, A consensus-based global optimization method for high dimensional machine learning problems, https://arxiv.org/abs/1909.09249v1 ] without resorting to the corresponding mean-field model. Our convergence analysis consists of two steps. In the first step, we show that the CBO model exhibits a global consensus time asymptotically for any initial data, and in the second step, we provide a sufficient condition on system parameters — which is dimension independent — and initial data which guarantee that the converged consensus state lies in a small neighborhood of the global minimum almost surely.

Convergence Toward Equilibrium of the First-Order Consensus Model with Random Batch Interactions

Uniform Error Estimates for the Random Batch Method to the First‐order Consensus Models with Antisymmetric Interaction Kernels

Convergence analysis of the discrete consensus-based optimization algorithm with random batch interactions and heterogeneous noises

Uniform-in-time Error Estimate of the Random Batch Method for the Cucker–Smale Model

On the Random Batch Method for second order interacting particle systems

Convergence of Random Batch Method with replacement for interacting particle systems

Convergence of Random Batch Method for Interacting Particles with Disparate Species and Weights

Convergence of Random Batch Method for interacting particles with disparate species and weights

Random Batch Method with Momentum Correction

On the mean-field limit of the Cucker-Smale model with Random Batch Method

Convergence of a First-Order Consensus-Based Global Optimization Algorithm

Mean field error estimate of the random batch method for large interacting particle system

Consensus Convergence with Stochastic Effects

Some remarks on the effect of the Random Batch Method on phase transition

Convergence Rate Of The Modified Degroot-Friedkin Model With Doubly Stochastic Relative Interaction Matrices

On Random Batch Methods (RBM) for interacting particle systems driven by Lévy processes

Ergodicity and long-time behavior of the Random Batch Method for interacting particle systems

On Convergence to Equilibria for a Chemotaxis Model with Volume-Filling Effect.

Convergence analysis of an explicit method and its random batch approximation for the McKean-Vlasov equations with non-globally Lipschitz conditions

Convergence to equilibrium of renormalised solutions to nonlinear chemical reaction-diffusion systems