Abstract:In this paper we provide a rigorous convergence analysis for the renowned particle swarm optimization method by using tools from stochastic calculus and the analysis of partial differential equations. Based on a continuous-time formulation of the particle dynamics as a system of stochastic differential equations, we establish convergence to a global minimizer of a possibly nonconvex and nonsmooth objective function in two steps. First, we prove consensus formation of an associated mean-field dynamics by analyzing the time-evolution of the variance of the particle distribution, which acts as Lyapunov function of the dynamics. We then show that this consensus is close to a global minimizer by employing the asymptotic Laplace principle and a tractability condition on the energy landscape of the objective function. These results allow for the usage of memory mechanisms, and hold for a rich class of objectives provided certain conditions of well-preparation of the hyperparameters and the initial datum. In a second step, at least for the case without memory effects, we provide a quantitative result about the mean-field approximation of particle swarm optimization, which specifies the convergence of the interacting particle system to the associated mean-field limit. Combining these two results allows for global convergence guarantees of the numerical particle swarm optimization method with provable polynomial complexity. To demonstrate the applicability of the method we propose an efficient and parallelizable implementation, which is tested in particular on a competitive and well-understood high-dimensional benchmark problem in machine learning.

Convergence Analysis of Standard Particle Swarm Optimization Algorithm

Algebraic and Dynamic Analysis on the Modified Swarm Optimizers

The analysis of particle swarm optimization algorithm’s convergence

A Method of Parameter Optimization for Particle Swarm Optimization Based on Stochastic Processes

On the Global Convergence of Particle Swarm Optimization Methods

Convergence analysis of the particle swarm optimization with stochastic inertia weight

The Convergence Basis of Particle Swarm Optimization

The particle swarm optimization algorithm: convergence analysis and parameter selection

Analyzing Convergence and Rates of Convergence of Particle Swarm Optimization Algorithms Using Stochastic Approximation Methods

The Analysis and Implementation of Particle Swarm Algorithm Convergence in Probability

The Convergence Indicator: Improved and completely characterized parameter bounds for actual convergence of Particle Swarm Optimization

A New Particle Swarm Optimization Algorithm and Its Convergence Analysis

Convergence Analysis of Swarm Algorithm.

Particle Swarm Optimization Via Convergence-Divergence Mechanism

Convergence analysis of particle swarm optimization using stochastic Lyapunov functions and quantifier elimination

Novel global convergence stochastic particle swarm optimizers

The Convergence Analysis of the Uniform Search Particle Swarm Optimization

On convergence analysis of multi-objective particle swarm optimization algorithm

Random convergence analysis of particle swarm optimization algorithm with time‐varying attractor

Analysis of Standard Particle Swarm Optimization Algorithm Based on Markov Chain

A Modified Particle Swarm Optimizer With A Novel Operator