Abstract:Human brain and behavior provide a rich venue that can inspire novel control and learning methods for robotics. In an attempt to exemplify such a development by inspiring how humans acquire knowledge and transfer skills among tasks, we introduce a novel multi-task reinforcement learning framework named Episodic Return Progress with Bidirectional Progressive Neural Networks (ERP-BPNN). The proposed ERP-BPNN model 1) learns in a human-like interleaved manner by 2) autonomous task switching based on a novel intrinsic motivation signal and, in contrast to existing methods, 3) allows bidirectional skill transfer among tasks. ERP-BPNN is a general architecture applicable to several multi-task learning settings; in this paper, we present the details of its neural architecture and show its ability to enable effective learning and skill transfer among morphologically different robots in a reaching task. The developed Bidirectional Progressive Neural Network (BPNN) architecture enables bidirectional skill transfer without requiring incremental training and seamlessly integrates with online task arbitration. The task arbitration mechanism developed is based on soft Episodic Return progress (ERP), a novel intrinsic motivation (IM) signal. To evaluate our method, we use quantifiable robotics metrics such as 'expected distance to goal' and 'path straightness' in addition to the usual reward-based measure of episodic return common in reinforcement learning. With simulation experiments, we show that ERP-BPNN achieves faster cumulative convergence and improves performance in all metrics considered among morphologically different robots compared to the baselines. Overall, our method provides a human-inspired and efficient multi-task reinforcement learning approach with interleaved learning, making it highly suitable for lifelong learning applications.

Transferring Meta-Policy from Simulation to Reality via Progressive Neural Network

Sim-to-Real Policy and Reward Transfer with Adaptive Forward Dynamics Model

Generalize Robot Learning from Demonstration to Variant Scenarios with Evolutionary Policy Gradient

Leveraging the Efficiency of Multi-Task Robot Manipulation Via Task-Evoked Planner and Reinforcement Learning

Modular Deep Q Networks for Sim-to-real Transfer of Visuo-motor Policies

Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning

Meta-Evolve: Continuous Robot Evolution for One-to-many Policy Transfer

A multi-robot path-planning algorithm for autonomous navigation using meta-reinforcement learning based on transfer learning

Sim-to-Real Transfer with Neural-Augmented Robot Simulation

Bidirectional Progressive Neural Networks With Episodic Return Progress for Emergent Task Sequencing and Robotic Skill Transfer

Policy Stitching: Learning Transferable Robot Policies

Robot Fleet Learning via Policy Merging

Transferring Multi-Agent Reinforcement Learning Policies for Autonomous Driving using Sim-to-Real

TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correction

Real–Sim–Real Transfer for Real-World Robot Control Policy Learning with Deep Reinforcement Learning

Efficient Deep Reinforcement Learning Through Policy Transfer.

One-shot sim-to-real transfer policy for robotic assembly via reinforcement learning with visual demonstration

Multi-Task Policy Search

A novel simulation reality closed loop learning framework for autonomous robot skill learning

Towards Closing the Sim-to-Real Gap in Collaborative Multi-Robot Deep Reinforcement Learning