Abstract:We focus on the challenge of finding a diverse collection of quality solutions on complex continuous domains. While quality diver-sity (QD) algorithms like Novelty Search with Local Competition (NSLC) and MAP-Elites are designed to generate a diverse range of solutions, these algorithms require a large number of evaluations for exploration of continuous spaces. Meanwhile, variants of the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) are among the best-performing derivative-free optimizers in single-objective continuous domains. This paper proposes a new QD algorithm called Covariance Matrix Adaptation MAP-Elites (CMA-ME). Our new algorithm combines the self-adaptation techniques of CMA-ES with archiving and mapping techniques for maintaining diversity in QD. Results from experiments based on standard continuous optimization benchmarks show that CMA-ME finds better-quality solutions than MAP-Elites; similarly, results on the strategic game Hearthstone show that CMA-ME finds both a higher overall quality and broader diversity of strategies than both CMA-ES and MAP-Elites. Overall, CMA-ME more than doubles the performance of MAP-Elites using standard QD performance metrics. These results suggest that QD algorithms augmented by operators from state-of-the-art optimization algorithms can yield high-performing methods for simultaneously exploring and optimizing continuous search spaces, with significant applications to design, testing, and reinforcement learning among other domains.

Natural Gradient Interpretation of Rank-One Update in CMA-ES

A Simple Yet Efficient Rank One Update for Covariance Matrix Adaptation.

An Efficient Rank-1 Update for Cholesky CMA-ES Using Auxiliary Evolution Path.

Covariance Matrix Adaptation Evolution Strategy for Low Effective Dimensionality

CMA-ES with Learning Rate Adaptation

The CMA Evolution Strategy: A Tutorial

CMA-ES with Adaptive Reevaluation for Multiplicative Noise

BCMA-ES: A Bayesian approach to CMA-ES

(1+1)-CMA-ES with Margin for Discrete and Mixed-Integer Problems

CMA-ES with Learning Rate Adaptation: Can CMA-ES with Default Population Size Solve Multimodal and Noisy Problems?

Covariance Matrix Adaptation MAP-Annealing

Covariance matrix adaptation for multi-objective optimization

Bound Analysis of Natural Gradient Descent in Stochastic Optimization Setting

Covariance Matrix Adaptation for the Rapid Illumination of Behavior Space

Model Complex Control CMA-ES.

Fast Moving Natural Evolution Strategy for High-Dimensional Problems

On-line Search History-assisted Restart Strategy for Covariance Matrix Adaptation Evolution Strategy

Large-Scale Evolution Strategy Based on Search Direction Adaptation

Covariance matrix adaptation evolution strategy based on correlated evolution paths with application to reinforcement learning

An Adaptive Re-evaluation Method for Evolution Strategy under Additive Noise

Generalized Low-Rank Update: Model Parameter Bounds for Low-Rank Training Data Modifications