Abstract:As Artificial and Robotic Systems are increasingly deployed and relied upon for real-world applications, it is important that they exhibit the ability to continually learn and adapt in dynamically-changing environments, becoming Lifelong Learning Machines. Continual/lifelong learning (LL) involves minimizing catastrophic forgetting of old tasks while maximizing a model's capability to learn new tasks. This paper addresses the challenging lifelong reinforcement learning (L2RL) setting. Pushing the state-of-the-art forward in L2RL and making L2RL useful for practical applications requires more than developing individual L2RL algorithms; it requires making progress at the systems-level, especially research into the non-trivial problem of how to integrate multiple L2RL algorithms into a common framework. In this paper, we introduce the Lifelong Reinforcement Learning Components Framework (L2RLCF), which standardizes L2RL systems and assimilates different continual learning components (each addressing different aspects of the lifelong learning problem) into a unified system. As an instantiation of L2RLCF, we develop a standard API allowing easy integration of novel lifelong learning components. We describe a case study that demonstrates how multiple independently-developed LL components can be integrated into a single realized system. We also introduce an evaluation environment in order to measure the effect of combining various system components. Our evaluation environment employs different LL scenarios (sequences of tasks) consisting of Starcraft-2 minigames and allows for the fair, comprehensive, and quantitative comparison of different combinations of components within a challenging common evaluation environment.

Wield: Systematic Reinforcement Learning With Progressive Randomization

Hand-in-Hand Guidance: an Explore-Exploit Based Reinforcement Learning Method for Performance Driven Assembly-Adjustment

A Multi-Task Approach to Robust Deep Reinforcement Learning for Resource Allocation

Continual Domain Randomization

Challenges of real-world reinforcement learning: definitions, benchmarks and analysis

Generating Automatic Curricula via Self-Supervised Active Domain Randomization

Curriculum Design and Sim2Real Transfer for Reinforcement Learning in Robotic Dual-Arm Assembly

Robust Multi-Modal Policies for Industrial Assembly via Reinforcement Learning and Demonstrations: A Large-Scale Study

Reward Augmentation in Reinforcement Learning for Testing Distributed Systems

A Large Language Model-Driven Reward Design Framework via Dynamic Feedback for Reinforcement Learning

Real-World Dexterous Object Manipulation based Deep Reinforcement Learning

System Design for an Integrated Lifelong Reinforcement Learning Agent for Real-Time Strategy Games

Reinforcement Learning on Web Interfaces Using Workflow-Guided Exploration

Modular Architecture for StarCraft II with Deep Reinforcement Learning

Domain Randomization for Robust, Affordable and Effective Closed-loop Control of Soft Robots

Towards robust and domain agnostic reinforcement learning competitions

Robust Deep Reinforcement Learning Scheduling via Weight Anchoring

Programmatic Reward Design by Example

Modular Robot Design Synthesis with Deep Reinforcement Learning

Powderworld: A Platform for Understanding Generalization via Rich Task Distributions

Active Task Randomization: Learning Robust Skills via Unsupervised Generation of Diverse and Feasible Tasks