Abstract:Deep reinforcement learning (RL) provides powerful methods for training optimal sequential decision-making agents. As collecting real-world interactions can entail additional costs and safety risks, the common paradigm of sim2real conducts training in a simulator, followed by real-world deployment. Unfortunately, RL agents easily overfit to the choice of simulated training environments, and worse still, learning ends when the agent masters the specific set of simulated environments. In contrast, the real world is highly open-ended, featuring endlessly evolving environments and challenges, making such RL approaches unsuitable. Simply randomizing over simulated environments is insufficient, as it requires making arbitrary distributional assumptions and can be combinatorially less likely to sample specific environment instances that are useful for learning. An ideal learning process should automatically adapt the training environment to maximize the learning potential of the agent over an open-ended task space that matches or surpasses the complexity of the real world. This thesis develops a class of methods called Unsupervised Environment Design (UED), which aim to produce such open-ended processes. Given an environment design space, UED automatically generates an infinite sequence or curriculum of training environments at the frontier of the learning agent's capabilities. Through extensive empirical studies and theoretical arguments founded on minimax-regret decision theory and game theory, the findings in this thesis show that UED autocurricula can produce RL agents exhibiting significantly improved robustness and generalization to previously unseen environment instances. Such autocurricula are promising paths toward open-ended learning systems that achieve more general intelligence by continually generating and mastering additional challenges of their own design.

Subjective Learning for Open-Ended Data

A Local Online Learning Approach for Non-linear Data.

A Definition of Open-Ended Learning Problems for Goal-Conditioned Agents

Learning Bounds for Open-Set Learning

Learning Curricula in Open-Ended Worlds

Open-set learning under covariate shift

OPEn: An Open-ended Physics Environment for Learning Without a Task

Multi-Task Curriculum Framework for Open-Set Semi-Supervised Learning

Contrastive Open Set Recognition

Open Set Domain Adaptation: Theoretical Bound and Algorithm

Self-Supervised Features Improve Open-World Learning

Dynamic Against Dynamic: An Open-set Self-learning Framework

Learning Object State Changes in Videos: An Open-World Perspective

Towards Realistic Long-tailed Semi-supervised Learning in an Open World

A Graph-Theoretic Framework for Understanding Open-World Semi-Supervised Learning

Rethinking Open-World Semi-Supervised Learning: Distribution Mismatch and Inductive Inference

Learning Class and Domain Augmentations for Single-Source Open-Domain Generalization

A Review of Open-World Learning and Steps Toward Open-World Learning Without Labels

An Empirical Study and Analysis on Open-Set Semi-Supervised Learning

Open-Set Semi-Supervised Learning for 3D Point Cloud Understanding

Universal Model Adaptation by Style Augmented Open-set Consistency