Abstract:Novelty search (NS) refers to a class of exploration algorithms that automatically uncover diverse system behaviors through simulations or experiments. Systematically obtaining diverse outcomes is a key component in many real-world design problems such as material and drug discovery, neural architecture search, reinforcement learning, and robot navigation. Since the relationship between the inputs and outputs (i.e., behaviors) of these complex systems is typically not available in closed form, NS requires a black-box perspective. Consequently, popular NS algorithms rely on evolutionary optimization and other meta-heuristics that require intensive sampling of the input space, which is impractical when the system is expensive to evaluate. We propose a Bayesian optimization inspired algorithm for sample-efficient NS that is specifically designed for such expensive black-box systems. Our approach models the input-to-behavior mapping with multi-output Gaussian processes (MOGP) and selects the next point to evaluate by maximizing a novelty metric that depends on a posterior sample drawn from the MOGP that promotes both exploration and exploitation. By leveraging advances in efficient posterior sampling and high-dimensional Gaussian process modeling, we discuss how our approach can be made scalable with respect to both amount of data and number of inputs. We test our approach on ten synthetic benchmark problems and eight real-world problems (with up to 2133 inputs) including new applications such as discovery of diverse metal organic frameworks for use in clean energy technology. We show that our approach greatly outperforms existing NS algorithms by finding substantially larger sets of diverse behaviors under limited sample budgets.

Efficiently evolving programs through the search for novelty

Evolving plastic neural networks with novelty search

Efficient Novelty Search Through Deep Reinforcement Learning

Combining Neuroevolution with the Search for Novelty to Improve the Generation of Test Inputs for Games

Evolution of Swarm Robotics Systems with Novelty Search

An Examination of Different Fitness and Novelty Based Selection Methods for the Evolution of Neural Networks

Comparing and Combining Lexicase Selection and Novelty Search

Guiding Genetic Programming with Graph Neural Networks

On Evolvability and Behavior Landscapes in Neuroevolutionary Divergent Search

Novelty Driven Evolutionary Neural Architecture Search

The Elephant in the Room: Towards the Application of Genetic Programming to Automatic Programming

Evolving black-box search algorithms employing genetic programming

Imperative Genetic Programming

Searching Search Spaces: Meta-evolving a Geometric Encoding for Neural Networks

(Genetically) Improving Novelty in Procedural Story Generation

BEACON: A Bayesian Optimization Strategy for Novelty Search in Expensive Black-Box Systems

Gene Expression Programming: a New Adaptive Algorithm for Solving Problems

Pyramid Search: Finding Solutions for Deceptive Problems Quickly in Genetic Programming

Discovering Evolutionary Stepping Stones through Behavior Domination

Challenges of ELA-guided Function Evolution using Genetic Programming

Iterative Genetic Improvement: Scaling Stochastic Program Synthesis