Abstract: Imitation learning is a class of promising policy learning algorithms that is free from many practical issues with reinforcement learning, such as the reward design issue and the exploration hardness. However, the current imitation algorithm struggles to achieve both high performance and high in-environment sample efficiency simultaneously. Behavioral Cloning (BC) does not need in-environment interactions, but it suffers from the covariate shift problem which harms its performance. Adversarial Imitation Learning (AIL) turns imitation learning into a distribution matching problem. It can achieve better performance on some tasks but it requires a large number of in-environment interactions. Inspired by the recent success of EfficientZero in RL, we propose EfficientImitate (EI), a planning-based imitation learning method that can achieve high in-environment sample efficiency and performance simultaneously. Our algorithmic contribution in this paper is two-fold. First, we extend AIL into the MCTS-based RL. Second, we show the seemingly incompatible two classes of imitation algorithms (BC and AIL) can be naturally unified under our framework, enjoying the benefits of both. We benchmark our method not only on the state-based DeepMind Control Suite, but also on the image version which many previous works find highly challenging. Experimental results show that EI achieves state-of-the-art results in performance and sample efficiency. EI shows over 4x gain in performance in the limited sample setting on state-based and image-based tasks and can solve challenging problems like Humanoid, where previous methods fail with small amount of interactions. Our code is available at https://github.com/zhaohengyin/EfficientImitate.

Efficient Imitation Learning for Game AI.

The Game Imitation: Deep Supervised Convolutional Networks for Quick Video Game AI

Planning for Sample Efficient Imitation Learning

Robust Imitation Learning for Automated Game Testing

Towards Informed Design and Validation Assistance in Computer Games Using Imitation Learning

Mimicking To Dominate: Imitation Learning Strategies for Success in Multiagent Competitive Games

Learning Strategy Representation for Imitation Learning in Multi-Agent Games

Sample-efficient Adversarial Imitation Learning from Observation

Game AI Competitions: Motivation for the Imitation Game-Playing Competition

Imitator Learning: Achieve Out-of-the-Box Imitation Ability in Variable Environments

MimicPlay: Long-Horizon Imitation Learning by Watching Human Play

Inspector: Pixel-Based Automated Game Testing via Exploration, Detection, and Investigation

Automatically Reinforcing a Game AI

Preference-conditioned Pixel-based AI Agent For Game Testing

Discovering of Game AIs' Characters Using a Neural Network Based AI Imitator for AI Clustering.

Resolving Copycat Problems in Visual Imitation Learning Via Residual Action Prediction

Imitation Learning from Imperfection: Theoretical Justifications and Algorithms

Imitation Learning Approach for AI Driving Olympics Trained on Real-world and Simulation Data Simultaneously

Improving Generalization in Game Agents with Data Augmentation in Imitation Learning

How To Guide Your Learner: Imitation Learning with Active Adaptive Expert Involvement

Keyframe-Focused Visual Imitation Learning