Abstract:Navigating deceptive domains has often been a challenge in machine learning due to search algorithms getting stuck at sub-optimal local optima. Many algorithms have been proposed to navigate these domains by explicitly maintaining diversity or equivalently promoting exploration, such as Novelty Search or other so-called Quality Diversity algorithms. In this paper, we present an approach with promise to solve deceptive domains without explicit diversity maintenance by optimizing a potentially large set of defined objectives. These objectives can be extracted directly from the environment by sub-aggregating the raw performance of individuals in a variety of ways. We use lexicase selection to optimize for these objectives as it has been shown to implicitly maintain population diversity. We compare this technique with a varying number of objectives to a commonly used quality diversity algorithm, MAP-Elites, on a set of discrete optimization as well as reinforcement learning domains with varying degrees of deception. We find that decomposing objectives into many objectives and optimizing them outperforms MAP-Elites on the deceptive domains that we explore. Furthermore, we find that this technique results in competitive performance on the diversity-focused metrics of QD-Score and Coverage, without explicitly optimizing for these things. Our ablation study shows that this technique is robust to different subaggregation techniques. However, when it comes to non-deceptive, or ``illumination" domains, quality diversity techniques generally outperform our objective-based framework with respect to exploration (but not exploitation), hinting at potential directions for future work.

Dynamic diversity is the answer to proxy failure

Proxy failure in academia: More than just another example

Teleonomy, legibility, and diversity: Do we need more "proxynomics"?

Proxy failures in practice: Examples from the sociology of science

The impact of diversity on group decision-making in the face of the free-rider problem

Modelling Behavioural Diversity for Learning in Open-Ended Games

The dynamics of diversity on corporate boards

Proving and Rewarding Client Diversity to Strengthen Resilience of Blockchain Networks

Object Proxy Patterns for Accelerating Distributed Applications

Phasic Diversity Optimization for Population-Based Reinforcement Learning

Proxy Non-Discrimination in Data-Driven Systems

Diversity Empowers Intelligence: Integrating Expertise of Software Engineering Agents

Objectives Are All You Need: Solving Deceptive Problems Without Explicit Diversity Maintenance

Preparing for the Future -- Rethinking Proxy Apps

Elitism-based transfer learning and diversity maintenance for dynamic multi-objective optimization

Augmenting team diversity and performance by enabling agency and fairness criteria in recommendation algorithms

Interactive diversity disrupts cyclic dominance but maintains cooperation in spatial social dilemma games

Controlling Behavioral Diversity in Multi-Agent Reinforcement Learning

Proxy Selection in Transitive Proxy Voting

Strategic Proxy Voting on the Line

Diversification Across Mining Pools: Optimal Mining Strategies under PoW