Abstract:Generative Adversarial Imitation Learning (GAIL) can learn policies without explicitly defining the reward function from demonstrations. GAIL has the potential to learn policies with high-dimensional observations as input, e.g., images. By applying GAIL to a real robot, perhaps robot policies can be obtained for daily activities like washing, folding clothes, cooking, and cleaning. However, human demonstration data are often imperfect due to mistakes, which degrade the performance of the resulting policies. We address this issue by focusing on the following features: 1) many robotic tasks are goal-reaching tasks, and 2) labeling such goal states in demonstration data is relatively easy. With these in mind, this paper proposes Goal-Aware Generative Adversarial Imitation Learning (GA-GAIL), which trains a policy by introducing a second discriminator to distinguish the goal state in parallel with the first discriminator that indicates the demonstration data. This extends a standard GAIL framework to more robustly learn desirable policies even from imperfect demonstrations through a goal-state discriminator that promotes achieving the goal state. Furthermore, GA-GAIL employs the Entropy-maximizing Deep P-Network (EDPN) as a generator, which considers both the smoothness and causal entropy in the policy update, to achieve stable policy learning from two discriminators. Our proposed method was successfully applied to two real-robotic cloth-manipulation tasks: turning a handkerchief over and folding clothes. We confirmed that it learns cloth-manipulation policies without task-specific reward function design. Video of the real experiments are available at <a class="link-external link-https" href="https://youtu.be/h_nII2ooUrE" rel="external noopener nofollow">this https URL</a>.

Generative Adversarial Imitation Learning from Failed Experiences

CEIL: Generalized Contextual Imitation Learning

On Computation and Generalization of Generative Adversarial Imitation Learning.

Distributional generative adversarial imitation learning with reproducing kernel generalization

Goal-Aware Generative Adversarial Imitation Learning from Imperfect Demonstration for Robotic Cloth Manipulation

GAILPG: Multi-Agent Policy Gradient with Generative Adversarial Imitation Learning

Improve generated adversarial imitation learning with reward variance regularization

A Pragmatic Look at Deep Imitation Learning

Exploring Gradient Explosion in Generative Adversarial Imitation Learning: A Probabilistic Perspective

Triple-GAIL: A Multi-Modal Imitation Learning Framework with Generative Adversarial Nets

C-GAIL: Stabilizing Generative Adversarial Imitation Learning with Control Theory

Diffusion-Reward Adversarial Imitation Learning

Non-Adversarial Imitation Learning and its Connections to Adversarial Methods

Latent Policies for Adversarial Imitation Learning

Wasserstein Distance guided Adversarial Imitation Learning with Reward Shape Exploration

Acgail: Imitation Learning About Multiple Intentions With Auxiliary Classifier Gans

Situated GAIL: Multitask imitation using task-conditioned adversarial inverse reinforcement learning

When Will Generative Adversarial Imitation Learning Algorithms Attain Global Convergence

Interpretable Generative Adversarial Imitation Learning

Fail-Safe Adversarial Generative Imitation Learning