Abstract:There has been substantial growth in research on the robot automation, which aims to make robots capable of directly interacting with the world or human. Robot learning for automation from human demonstration is central to such situation. However, the dependence of demonstration restricts robot to a fixed scenario, without the ability to explore in variant situations to accomplish the same task as in demonstration. Deep reinforcement learning methods may be a good method to make robot learning beyond human demonstration and fulfilling the task in unknown situations. The exploration is the core of such generalization to different environments. While the exploration in reinforcement learning may be ineffective and suffer from the problem of low sample efficiency. In this paper, we present Evolutionary Policy Gradient (EPG) to make robot learn from demonstration and perform goal oriented exploration efficiently. Through goal oriented exploration, our method can generalize robot learned skill to environments with different parameters. Our Evolutionary Policy Gradient combines parameter perturbation with policy gradient method in the framework of Evolutionary Algorithms (EAs) and can fuse the benefits of both, achieving effective and efficient exploration. With demonstration guiding the evolutionary process, robot can accelerate the goal oriented exploration to generalize its capability to variant scenarios. The experiments, carried out in robot control tasks in OpenAI Gym with dense and sparse rewards, show that our EPG is able to provide competitive performance over the original policy gradient methods and EAs. In the manipulator task, our robot can learn to open the door with vision in environments which are different from where the demonstrations are provided.

Jonathan (2010) Real-world Transfer of Evolved Artificial Immune System Behaviours Between Small and Large Scale Robotic Platforms. Evolutionary Intelligence, 3 (3-4). Pp. 123-136

The Transfer of Evolved Artificial Immune System Behaviours Between Small and Large Scale Robotic Platforms

Generalize Robot Learning from Demonstration to Variant Scenarios with Evolutionary Policy Gradient

Whitbrook , Amanda and Aickelin , Uwe and Garibaldi , Jon ( 2007 ) Idiotypic Immune Networks in Mobile Robot Control

Environmental Adaptation of Robot Morphology and Control Through Real-World Evolution

Idiotypic Immune Networks in Mobile Robot Control

Evolving homogeneous neurocontrollers for a group of heterogeneous robots: coordinated motion, cooperation, and acoustic communication

An Idiotypic Immune Network as a Short Term Learning Architecture for Mobile Robots

Onboard Evolution of Understandable Swarm Behaviors

The Use of Probabilistic Systems to Mimic the Behaviour of Idiotypic AIS Robot Controllers

Mimicking the Behaviour of Idiotypic AIS Robot Controllers Using Probabilistic Systems

Emergent adaptive behaviour of GRN-controlled simulated robots in a changing environment

Evolution and learning in differentiable robots

Whitbrook, Amanda and Aickelin, Uwe and Garibaldi, Jonathan M. (2010) Two-timescale learning using idiotypic behaviour mediation for a navigating mobile

Beyond pheromones: evolving error-tolerant, flexible, and scalable ant-inspired robot swarms

Genetic Algorithm Seeding of Idiotypic Networks for Mobile-Robot Navigation

Two-timescale Learning Using Idiotypic Behaviour Mediation for a Navigating Mobile Robot.

Evolution of Embodied Intelligence

The Evolution of Cooperative Behaviours in Physically Heterogeneous Multi-Robot Systems

Transfer Dynamics in Emergent Evolutionary Curricula

Learning Locomotion Skills in Evolvable Robots