Abstract:Games have long played an important role in the development and further understanding of co-evolutionary learning systems. In particular, the search process in co-evolutionary learning is guided by strategic interactions between solutions in the population, which can be naturally framed as game-playing. We study two important issues in co-evolutionary learning–generalization performance and diversity–using games. On the one hand, one is concerned with the co-evolutionary learning of strategies with high generalization performance, that is, strategies that can outperform against a large number of test strategies (opponents) that may not be seen before during co-evolution. On the other hand, the other one is concerned with diversity levels in the population that may lead to the search of strategies with poor generalization performance. It is not known if there is a relationship between generalization and diversity in co-evolutionary learning. This paper investigates whether there is such a relationship in co-evolutionary learning through a detailed empirical study. We systematically investigate the impact of various diversity maintenance approaches on the generalization performance of co-evolutionary learning quantitatively using case studies. The problem of the iterated prisoner’s dilemma (IPD) game is considered. Unlike what was done in past studies, we can measure both the generalization performance and the diversity level of the population of evolved strategies. Results from our case studies show that the introduction and maintenance of diversity do not necessarily lead to the co-evolutionary learning of strategies with high generalization performance. However, if individual strategies can be combined in the form of ensembles, there is the potential of exploiting diversity in co-evolutionary learning to improve generalization performance. Specifically, when the introduction and maintenance of diversity lead to a speciated population during co-evolution, where each specialist strategy is capable of outperforming different opponents, the ensemble can have a significantly higher generalization performance compared to individual strategies when it can sufficiently often choose the best specialist strategy against a test strategy.

Exploiting Coalition In Co-Evolutionary Learning

Evolutionary Learning of Multiagents Using Strategic Coalition in the IPD Game

Co-Evolutionary Learning Of Contextual Asymmetric Actors

Aspiration-driven co-evolution of cooperation with individual behavioral diversity

The influence of experienced guider on cooperative behavior in the Prisoner's dilemma game

Keynote speech I: Co-evolutionary learning in game-playing.

Multiple Choices and Reputation in Multiagent Interactions

Coevolution of Extortion Strategies with Mixed Imitation and Aspiration Learning Dynamics in Spatial Prisoner’s Dilemma Game

Effect of State Transition Triggered by Reinforcement Learning in Evolutionary Prisoner’s Dilemma Game

Learning dynamics explains human behavior in Prisoner's Dilemma on networks

Individual's expulsion to nasty environment promotes cooperation in public goods games

Learning Algorithm and the Cooperation Behavior of Continuous Prisoner's Dilemma Game on Complex Networks

Q-learning Boosts the Evolution of Cooperation in Structured Population by Involving Extortion

Using Social Network to Evolve Cooperation in Multiple Choices Iterated Prisoner’s Dilemma Game

Behavioral Diversity, Choices and Noise in the Iterated Prisoner's Dilemma

Evolving Cooperation in Complex Behavioral Interactions through Reputation Siang

Could feedback-based self-learning help solve networked Prisoner's Dilemma?

Evolutionary Game Dynamics of Multi-Agent Cooperation Driven by Self-Learning

The Evolution of Cooperation in Spatial Prisoner’s Dilemma Game with Dynamic Relationship-Based Preferential Learning

Generalization and Diversity in Coevolutionary Learning

Improving Generalization Performance in Co-Evolutionary Learning