Abstract:Efficiently tackling multiple tasks within complex environment, such as those found in robot manipulation, remains an ongoing challenge in robotics and an opportunity for data-driven solutions, such as reinforcement learning (RL). Model-based RL, by building a dynamic model of the robot, enables data reuse and transfer learning between tasks with the same robot and similar environment. Furthermore, data gathering in robotics is expensive and we must rely on data efficient approaches such as model-based RL, where policy learning is mostly conducted on cheaper simulations based on the learned model. Therefore, the quality of the model is fundamental for the performance of the posterior tasks. In this work, we focus on improving the quality of the model and maintaining the data efficiency by performing active learning of the dynamic model during a preliminary exploration phase based on maximize information gathering. We employ Bayesian neural network models to represent, in a probabilistic way, both the belief and information encoded in the dynamic model during exploration. With our presented strategies we manage to actively estimate the novelty of each transition, using this as the exploration reward. In this work, we compare several Bayesian inference methods for neural networks, some of which have never been used in a robotics context, and evaluate them in a realistic robot manipulation setup. Our experiments show the advantages of our Bayesian model-based RL approach, with similar quality in the results than relevant alternatives with much lower requirements regarding robot execution steps. Unlike related previous studies that focused the validation solely on toy problems, our research takes a step towards more realistic setups, tackling robotic arm end-tasks.

Data-Efficient Model-Based Reinforcement Learning For Robot Control

Robot Simulation and Reinforcement Learning Training Platform Based on Distributed Architecture.

Learning with Training Wheels: Speeding up Training with a Simple Controller for Deep Reinforcement Learning

Model-Based Reinforcement Learning for Robotic Arm Control with Limited Environment Interaction

Comparison of Model-Based and Model-Free Reinforcement Learning for Real-World Dexterous Robotic Manipulation Tasks

Model Learning for Two-Wheeled Robot Self-Balance Control

A Path-Integral-Based Reinforcement Learning Algorithm for Path Following of an Autoassembly Mobile Robot

RL + Model-based Control: Using On-demand Optimal Control to Learn Versatile Legged Locomotion

Guiding Reinforcement Learning with Incomplete System Dynamics

Residual Reinforcement Learning for Robot Control

Data Informed Residual Reinforcement Learning for High-Dimensional Robotic Tracking Control

Survey of Model-Based Reinforcement Learning: Applications on Robotics

SAM-RL: Sensing-Aware Model-Based Reinforcement Learning via Differentiable Physics-Based Simulation and Rendering

Real–Sim–Real Transfer for Real-World Robot Control Policy Learning with Deep Reinforcement Learning

Active Exploration in Bayesian Model-based Reinforcement Learning for Robot Manipulation

A Survey of Sim-to-Real Transfer Techniques Applied to Reinforcement Learning for Bioinspired Robots

Reinforcement learning building control approach harnessing imitation learning

Deep Incremental Model Based Reinforcement Learning: A One-Step Lookback Approach for Continuous Robotics Control