A Survey of Imitation Learning Methods, Environments and Metrics

Nathan Gavenski,Felipe Meneguzzi,Michael Luck,Odinaldo Rodrigues
2024-07-30
Abstract:Imitation learning is an approach in which an agent learns how to execute a task by trying to mimic how one or more teachers perform it. This learning approach offers a compromise between the time it takes to learn a new task and the effort needed to collect teacher samples for the agent. It achieves this by balancing learning from the teacher, who has some information on how to perform the task, and deviating from their examples when necessary, such as states not present in the teacher samples. Consequently, the field of imitation learning has received much attention from researchers in recent years, resulting in many new methods and applications. However, with this increase in published work and past surveys focusing mainly on methodology, a lack of standardisation became more prominent in the field. This non-standardisation is evident in the use of environments, which appear in no more than two works, and evaluation processes, such as qualitative analysis, that have become rare in current literature. In this survey, we systematically review current imitation learning literature and present our findings by (i) classifying imitation learning techniques, environments and metrics by introducing novel taxonomies; (ii) reflecting on main problems from the literature; and (iii) presenting challenges and future directions for researchers.
Machine Learning,Artificial Intelligence
What problem does this paper attempt to address?
The main problem this paper attempts to address is the lack of standardization in the current field of imitation learning, which leads to inconsistencies in the comparison and evaluation of different methods and applications. Specifically, with the rapid increase in imitation learning research, many new experimental environments and performance metrics have emerged, making it difficult to meaningfully compare different methods. Therefore, the authors systematically review the imitation learning literature and propose a new taxonomy aimed at: 1. **Introducing a new taxonomy**: These taxonomies focus on key aspects of imitation learning methods, environments, and metrics to provide a more unified and systematic framework. 2. **Discussing important issues**: Reflecting on and discussing important issues observed in the existing literature, such as the diversity and complexity of methods, the diversity of environments, and the inconsistency of evaluation metrics. 3. **Proposing future directions**: Highlighting open challenges and suggesting future research directions to promote further development in the field of imitation learning. Through these efforts, the paper provides a solid foundation for understanding and developing new imitation learning methods and promotes more systematic and consistent comparisons of technologies within the field.