Abstract:Estimation of distribution algorithms (EDAs) are widely used in stochastic optimization. Impressive experimental results have been reported in the literature. However, little work has been done on analyzing the computation time of EDAs in relation to the problem size. It is still unclear how well EDAs (with a finite population size larger than two) will scale up when the dimension of the optimization problem (problem size) goes up. This paper studies the computational time complexity of a simple EDA, i.e., the univariate marginal distribution algorithm (UMDA), in order to gain more insight into EDAs complexity. First, we discuss how to measure the computational time complexity of EDAs. A classification of problem hardness based on our discussions is then given. Second, we prove a theorem related to problem hardness and the probability conditions of EDAs. Third, we propose a novel approach to analyzing the computational time complexity of UMDA using discrete dynamic systems and Chernoff bounds. Following this approach, we are able to derive a number of results on the first hitting time of UMDA on a well-known unimodal pseudo-boolean function, i.e., the LeadingOnes problem, and another problem derived from LeadingOnes, named BVLeadingOnes. Although both problems are unimodal, our analysis shows that LeadingOnes is easy for the UMDA, while BVLeadingOnes is hard for the UMDA. Finally, in order to address the key issue of what problem characteristics make a problem hard for UMDA, we discuss in depth the idea of ¿margins¿ (or relaxation). We prove theoretically that the UMDA with margins can solve the BVLeadingOnes problem efficiently.

Reducing computational complexity of estimating multivariate histogram-based probabilistic model

Optimizing continuous problems using estimation of distribution algorithm based on histogram model

Histogram-Based Estimation of Distribution Algorithm: A Competent Method for Continuous Optimization.

An Improved Algorithm for High-Dimensional Continuous Threshold Expectile Model with Variance Heterogeneity

Nonparametric Density Estimation of Hierarchical Probabilistic Graph Models for Assumption-Free Monitoring

Gaussian mixture density modeling and decomposition with weighted likelihood

Process Monitoring Using Probabilistic Graphical Models Via Nonparametric Density Estimation

A Novel Fuzzy Histogram Based Estimation of Distribution Algorithm for Global Numerical Optimization.

Mp-Eda: A Robust Estimation Of Distribution Algorithm With Multiple Probabilistic Models For Global Continuous Optimization

A Bayesian View on the Polynomial Distribution Model in Estimation of Distribution Algorithms

Continuous optimization based-on greedy estimation of GMM

A Continuous Optimization Algorithm Based-on Progressive Estimation of Guassian Mixture Model

Analysis of Computational Time of Simple Estimation of Distribution Algorithms

An Estimation of Distribution Algorithm Based on Nonparametric Density Estimation

A Two-Level Hierarchical EDA Using Conjugate Priori.

A Mathematical Modelling Technique for the Analysis of the Dynamics of A Simple Continuous Eda

Marginal Probability Distribution Estimation In Characteristic Space Of Covariance-Matrix

Complex-valued Bayesian parameter estimation via Markov chain Monte Carlo

Frequentist model averaging for undirected gaussian graphical models

Probabilistic Modeling for Continuous EDA with Boltzmann Selection and Kullback-Leibeler Divergence