Abstract:It has been hypothesized that one of the main reasons evolution has been able to produce such impressive adaptations is because it has improved its own ability to evolve -- "the evolution of evolvability". Rupert Riedl, for example, an early pioneer of evolutionary developmental biology, suggested that the evolution of complex adaptations is facilitated by a developmental organization that is itself shaped by past selection to facilitate evolutionary innovation. However, selection for characteristics that enable future innovation seems paradoxical: natural selection cannot favor structures for benefits they have not yet produced, and favoring characteristics for benefits that have already been produced does not constitute future innovation. Here we resolve this paradox by exploiting a formal equivalence between the evolution of evolvability and learning systems. We use the conditions that enable simple learning systems to generalize, i.e., to use past experience to improve performance on previously unseen, future test cases, to demonstrate conditions where natural selection can systematically favor developmental organizations that benefit future evolvability. Using numerical simulations of evolution on highly epistatic fitness landscapes, we illustrate how the structure of evolved gene regulation networks can result in increased evolvability capable of avoiding local fitness peaks and discovering higher fitness phenotypes. Our findings support Riedl's intuition: Developmental organizations that "mimic" the organization of constraints on phenotypes can be favored by short-term selection and also facilitate future innovation. Importantly, the conditions that enable the evolution of such surprising evolvability follow from the same non-mysterious conditions that permit generalization in learning systems.

On Evolvability and Behavior Landscapes in Neuroevolutionary Divergent Search

An Examination of Different Fitness and Novelty Based Selection Methods for the Evolution of Neural Networks

Evolving plastic neural networks with novelty search

Evolving the Behavior of Machines: From Micro to Macroevolution

Resolving the paradox of evolvability with learning theory: How evolution learns to improve evolvability on rugged fitness landscapes

Evolution of Swarm Robotics Systems with Novelty Search

Efficient Novelty Search Through Deep Reinforcement Learning

Efficiently evolving programs through the search for novelty

Evolutionary search agents in complex landscapes - a new model for the role of competence and meta-competence (EVOLINO and other simulation tools)

Combining Neuroevolution with the Search for Novelty to Improve the Generation of Test Inputs for Games

Life, uh, Finds a Way: Systematic Neural Search

Long-Term Progress and Behavior Complexification in Competitive Co-Evolution

Dynamics-aware Novelty Search with Behavior Repulsion

Discovering Evolutionary Stepping Stones through Behavior Domination

Assessing Quality-Diversity Neuro-Evolution Algorithms Performance in Hard Exploration Problems

Efficient and Diverse Generative Robot Designs using Evolution and Intrinsic Motivation

Selecting for Selection: Learning To Balance Adaptive and Diversifying Pressures in Evolutionary Search

Effects of Different Optimization Formulations in Evolutionary Reinforcement Learning on Diverse Behavior Generation

Policy Manifold Search for Improving Diversity-based Neuroevolution

Surprise Search for Evolutionary Divergence

Improving Exploration in Evolution Strategies for Deep Reinforcement Learning via a Population of Novelty-Seeking Agents