Abstract:Estimation of Distribution Algorithms (EDAs) are stochastic heuristics that search for optimal solutions by learning and sampling from probabilistic models. Despite their popularity in real-world applications, there is little rigorous understanding of their performance. Even for the Univariate Marginal Distribution Algorithm (UMDA)—a simple population-based EDA assuming independence between decision variables—the optimisation time on the linear problem OneMax was until recently undetermined. The incomplete theoretical understanding of EDAs is mainly due to the lack of appropriate analytical tools. We show that the recently developed level-based theorem for non-elitist populations combined with anti-concentration results yield upper bounds on the expected optimisation time of the UMDA. This approach results in the bound Onλlogλ+n2\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\mathcal {O}\left( n\lambda \log \lambda +n^2\right) $$\end{document} on the LeadingOnes and BinVal problems for population sizes λ>μ=Ω(logn)\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\lambda >\mu =\varOmega (\log n)$$\end{document}, where μ\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\mu $$\end{document} and λ\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\lambda $$\end{document} are parameters of the algorithm. We also prove that the UMDA with population sizes μ∈On∩Ω(logn)\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\mu \in \mathcal {O}\left( \sqrt{n}\right) \cap \varOmega (\log n)$$\end{document} optimises OneMax in expected time Oλn\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\mathcal {O}\left( \lambda n\right) $$\end{document}, and for larger population sizes μ=Ω(nlogn)\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\mu =\varOmega (\sqrt{n}\log n)$$\end{document}, in expected time Oλn\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\mathcal {O}\left( \lambda \sqrt{n}\right) $$\end{document}. The facility and generality of our arguments suggest that this is a promising approach to derive bounds on the expected optimisation time of EDAs.

Convergence analysis of UMDAC with finite populations: a case study on flat landscapes

A Mathematical Modelling Technique for the Analysis of the Dynamics of A Simple Continuous Eda

Analysis of Computational Time of Simple Estimation of Distribution Algorithms

Convergence Analysis of MCMC Algorithms for Bayesian Multivariate Linear Regression with Non‐Gaussian Errors

Rigorous time complexity analysis of univariate marginal distribution algorithm with margins

A High-dimensional Convergence Theorem for U-statistics with Applications to Kernel-based Testing

Central limit theorems for high dimensional dependent data

Convergence analysis of data augmentation algorithms for Bayesian robust multivariate linear regression with incomplete data

Convergence of Dirichlet Forms for MCMC Optimal Scaling with Dependent Target Distributions on Large Graphs

Convergence of Unadjusted Langevin in High Dimensions: Delocalization of Bias

Wasserstein convergence for conditional empirical measures of subordinated Dirichlet diffusions on Riemannian manifolds

Lower bounds on the rate of convergence for accept-reject-based Markov chains in Wasserstein and total variation distances

Universal Inference Meets Random Projections: A Scalable Test for Log-concavity

Level-Based Analysis of the Univariate Marginal Distribution Algorithm

Convergence Analysis of Discrete Diffusion Model: Exact Implementation through Uniformization

Convergence rate of random scan Coordinate Ascent Variational Inference under log-concavity

An Error Analysis of Generative Adversarial Networks for Learning Distributions

Parallel Markov Chain Monte Carlo for Non-Gaussian Posterior Distributions

On the Analysis of Average Time Complexity of Estimation of Distribution Algorithms

Convergence Theory of Generalized Distributed Subgradient Method with Random Quantization

Statistical Convergence of the EM Algorithm on Gaussian Mixture Models