Abstract:The cross-entropy (CE) method is a popular stochastic method for optimization due to its simplicity and effectiveness. Designed for rare-event simulations where the probability of a target event occurring is relatively small, the CE-method relies on enough objective function calls to accurately estimate the optimal parameters of the underlying distribution. Certain objective functions may be computationally expensive to evaluate, and the CE-method could potentially get stuck in local minima. This is compounded with the need to have an initial covariance wide enough to cover the design space of interest. We introduce novel variants of the CE-method to address these concerns. To mitigate expensive function calls, during optimization we use every sample to build a surrogate model to approximate the objective function. The surrogate model augments the belief of the objective function with less expensive evaluations. We use a Gaussian process for our surrogate model to incorporate uncertainty in the predictions which is especially helpful when dealing with sparse data. To address local minima convergence, we use Gaussian mixture models to encourage exploration of the design space. We experiment with evaluation scheduling techniques to reallocate true objective function calls earlier in the optimization when the covariance is the largest. To test our approach, we created a parameterized test objective function with many local minima and a single global minimum. Our test function can be adjusted to control the spread and distinction of the minima. Experiments were run to stress the cross-entropy method variants and results indicate that the surrogate model-based approach reduces local minima convergence using the same number of function evaluations.

An Improved Multi-Objective Cross-Entropy Method For Inverse Problems

The Cross-Entropy Method and Its Application to Inverse Problems

An Improved Cross-Entropy Method Applied to Inverse Problems

Vector Design Optimizations Using an Improved Cross-Entropy Method

Multiobjective Optimization of Inverse Problems Using a Vector Cross Entropy Method

An Improved Regularity-Based Vector Evolutionary Algorithm for Multi-Objective Optimizations.

A Population-Based Incremental Learning Vector Algorithm For Multiobjective Optimal Designs

An Improved Physical Programming Method for Multi-Objective Inverse Problems

A Vector Quantum-Inspired Evolutionary Algorithm Applied to Multi-Objective Inverse Problems

Cross-Entropy Method Variants for Optimization

A Hybridized Vector Optimal Algorithm for Multiobjective Designs of Inverse Problems

Incorporating Light Beam Search in a Vector Normal Boundary Intersection Method for Multiobjective Inverse Problem

An Improved Normal Boundary Intersection Method for Multiobjective Inverse Problems

An Improved Population-Based Incremental Learning Method For Inverse Problems

An improved multi-objective differential evolution algorithm with an adaptive crossover rate

Adaptive Reference Vector Generation for Inverse Model Based Evolutionary Multiobjective Optimization with Degenerate and Disconnected Pareto Fronts

A Tutorial on the Cross-Entropy Method

Dynamic Reference Vectors and Biased Crossover Use for Inverse Model Based Evolutionary Multi-Objective Optimization with Irregular Pareto Fronts

Constrained Evolutionary Optimization by Means of (μ + Λ)-Differential Evolution and Improved Adaptive Trade-Off Model