Abstract:In this paper we provide a rigorous convergence analysis for the renowned particle swarm optimization method by using tools from stochastic calculus and the analysis of partial differential equations. Based on a continuous-time formulation of the particle dynamics as a system of stochastic differential equations, we establish convergence to a global minimizer of a possibly nonconvex and nonsmooth objective function in two steps. First, we prove consensus formation of an associated mean-field dynamics by analyzing the time-evolution of the variance of the particle distribution, which acts as Lyapunov function of the dynamics. We then show that this consensus is close to a global minimizer by employing the asymptotic Laplace principle and a tractability condition on the energy landscape of the objective function. These results allow for the usage of memory mechanisms, and hold for a rich class of objectives provided certain conditions of well-preparation of the hyperparameters and the initial datum. In a second step, at least for the case without memory effects, we provide a quantitative result about the mean-field approximation of particle swarm optimization, which specifies the convergence of the interacting particle system to the associated mean-field limit. Combining these two results allows for global convergence guarantees of the numerical particle swarm optimization method with provable polynomial complexity. To demonstrate the applicability of the method we propose an efficient and parallelizable implementation, which is tested in particular on a competitive and well-understood high-dimensional benchmark problem in machine learning.

Convergence analysis of sectional methods for solving aggregation population balance equations: The fixed pivot technique

Modified variational iteration method and its convergence analysis for solving nonlinear aggregation population balance equation

On the Convergence of the Accelerated Riccati Iteration Method.

Approximate solutions to the nonlinear hyperbolic population balance equation: convergence, error estimate and numerical simulations

Semi-Analytical Methods for Population Balance models involving Aggregation and Breakage processes: A comparative study

A new analytical solution for solving the population balance equation in the continuum-slip regime

Coupled approach and its convergence analysis for aggregation and breakage models: Study of extended temporal behaviour

Convergence Analysis of a Multigrid Method for a Nonlocal Model.

Consistency and Convergence Properties of the Isogeometric Collocation Method

The Convergence Rate and Necessary-and-Sufficient Condition for the Consistency of Isogeometric Collocation Method

Local Fixed Pivot Quadrature Method of Moments for Solution of Population Balance Equation

Convergence Proof for the Multigrid Method of the Nonlocal Model

On the Global Convergence of Particle Swarm Optimization Methods

Convergence Analysis on Multigrid Methods for Elliptic Problems with Large Jumps in Coefficients

Elzaki projected differential transform method for multi-dimensional aggregation and combined aggregation-breakage equations

Uniform Convergence of Adaptive Multigrid Methods for Elliptic Problems and Maxwell's Equations

An efficient computational technique and its convergence analysis for a class of doubly singular boundary value problems

Exact Solutions of Population Balance Equation with Aggregation, Nucleation, Growth and Breakage Processes, Using Scaling Group Analysis

The Convergence Analysis of a Class of Stabilized Semi-Implicit Isogeometric Methods for the Cahn-Hilliard Equation

Finite volume convergence analysis and error estimation for non-linear collisional induced breakage equation

Convergence Analysis For Non Linear System Of Parabolic Variational Inequalities