Abstract:There has been substantial growth in research on the robot automation, which aims to make robots capable of directly interacting with the world or human. Robot learning for automation from human demonstration is central to such situation. However, the dependence of demonstration restricts robot to a fixed scenario, without the ability to explore in variant situations to accomplish the same task as in demonstration. Deep reinforcement learning methods may be a good method to make robot learning beyond human demonstration and fulfilling the task in unknown situations. The exploration is the core of such generalization to different environments. While the exploration in reinforcement learning may be ineffective and suffer from the problem of low sample efficiency. In this paper, we present Evolutionary Policy Gradient (EPG) to make robot learn from demonstration and perform goal oriented exploration efficiently. Through goal oriented exploration, our method can generalize robot learned skill to environments with different parameters. Our Evolutionary Policy Gradient combines parameter perturbation with policy gradient method in the framework of Evolutionary Algorithms (EAs) and can fuse the benefits of both, achieving effective and efficient exploration. With demonstration guiding the evolutionary process, robot can accelerate the goal oriented exploration to generalize its capability to variant scenarios. The experiments, carried out in robot control tasks in OpenAI Gym with dense and sparse rewards, show that our EPG is able to provide competitive performance over the original policy gradient methods and EAs. In the manipulator task, our robot can learn to open the door with vision in environments which are different from where the demonstrations are provided.

GAILPG: Multi-Agent Policy Gradient with Generative Adversarial Imitation Learning

Generalize Robot Learning from Demonstration to Variant Scenarios with Evolutionary Policy Gradient

Independent Generative Adversarial Self-Imitation Learning in Cooperative Multiagent Systems

Dueling Network Architecture for Multi-Agent Deep Deterministic Policy Gradient

Efficient Off-policy Adversarial Imitation Learning with Imperfect Demonstrations

A Collaborative Multiagent Reinforcement Learning Method Based on Policy Gradient Potential

Distributional generative adversarial imitation learning with reproducing kernel generalization

Adversarial imitation learning with mixed demonstrations from multiple demonstrators

On Computation and Generalization of Generative Adversarial Imitation Learning.

A Pragmatic Look at Deep Imitation Learning

When Will Generative Adversarial Imitation Learning Algorithms Attain Global Convergence

Triple-GAIL: A Multi-Modal Imitation Learning Framework with Generative Adversarial Nets

C-GAIL: Stabilizing Generative Adversarial Imitation Learning with Control Theory

Exploring Gradient Explosion in Generative Adversarial Imitation Learning: A Probabilistic Perspective

Generative Adversarial Imitation Learning from Failed Experiences

Multi-Agent Deep Deterministic Policy Gradient Algorithm Based on Classification Experience Replay

Situated GAIL: Multitask imitation using task-conditioned adversarial inverse reinforcement learning

Taking Complementary Advantages: Improving Exploration Via Double Self-Imitation Learning in Procedurally-Generated Environments

Adversarial Imitation Learning from Incomplete Demonstrations

CPIG: Leveraging Consistency Policy with Intention Guidance for Multi-agent Exploration

Wasserstein Distance guided Adversarial Imitation Learning with Reward Shape Exploration