Uniform-in-time mean-field limit estimate for the Consensus-Based Optimization

Hui Huang,Hicham Kouhkouh

2024-11-06

Abstract:We establish a uniform-in-time estimate for the mean-field convergence of the Consensus-Based Optimization (CBO) algorithm by rescaling the consensus point in the dynamics with a small parameter $\kappa \in (0,1)$. This uniform-in-time estimate is essential, as CBO convergence relies on a sufficiently large time horizon and is crucial for ensuring stable, reliable long-term convergence, the latter being key to the practical effectiveness of CBO methods.

Optimization and Control,Probability

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to establish the mean - field limit estimate of the Consensus - Based Optimization (CBO) over a long period of time. Specifically, the author rescales the consensus points and introduces a small parameter \(\kappa\in(0, 1)\) to ensure that the convergence of the CBO algorithm remains stable and reliable over a sufficiently long time range.

### Problem Background
The Consensus - Based Optimization (CBO) is an effective method for solving non - convex and non - smooth optimization problems. It simulates the collective behavior in social and biological systems and finds the optimal solution through the interaction between multiple agents or particles. However, most of the existing research on the convergence of CBO focuses on the mean - field limit estimate within a finite time range. These estimates rely on the exponential constant of time \(T\) and cannot provide stability guarantees over a long - time range.

### Research Motivation
To ensure the effectiveness and reliability of the CBO algorithm in practical applications, it is necessary to prove its convergence over a long - time range. Therefore, this paper aims to establish a **uniform - in - time mean - field limit estimate** to ensure that the CBO algorithm can still converge stably over a long - time range.

### Main Contributions
1. **Establishing a long - time stable mean - field limit estimate**: The author rescales the consensus points by introducing a small parameter \(\kappa\), which strengthens the constraint properties of the dynamic system and proves the existence of the mean - field limit over a long - time range.
2. **Theoretical analysis and numerical verification**: Through strict mathematical derivations and numerical experiments, the effectiveness of the proposed improved CBO algorithm is verified. In particular, this method shows good performance in high - dimensional optimization problems.

### Formula Summary
- Dynamic equation:
\[
dX_t=-\lambda(X_t - \kappa m_\alpha(\rho_t))dt+\sigma(\delta I_d + D(X_t - \kappa m_\alpha(\rho_t)))dB_t
\]
where \(m_\alpha(\rho_t)=\frac{\int_{\mathbb{R}^d}x\omega_f^\alpha(x)\rho_t(dx)}{\int_{\mathbb{R}^d}\omega_f^\alpha(x)\rho_t(dx)}\) is the current global consensus point, and the weight function is \(\omega_f^\alpha(x)=\exp(-\alpha f(x))\).

- Mean - field limit estimate:
\[
\sup_{t\geq0}\sup_{i = 1,\ldots,N}\mathbb{E}[|X_i^N(t)-X_i(t)|^2]\leq C_{MFL}N^{-1}
\]
where \(C_{MFL}\) is a time - independent constant that depends on \(\lambda, \sigma, \kappa, C_2, C_3\) and \(L\).

Through these improvements, the author has successfully solved the key problem of the stable convergence of the CBO algorithm over a long - time range, providing theoretical basis and technical support for further research in this field.