Improved Inference of Human Intent by Combining Plan Recognition and Language Feedback

Ifrah Idrees,Tian Yun,Naveen Sharma,Yunxin Deng,Nakul Gopalan,George Konidaris,Stefanie Tellex

2023-10-04

Abstract:Conversational assistive robots can aid people, especially those with cognitive impairments, to accomplish various tasks such as cooking meals, performing exercises, or operating machines. However, to interact with people effectively, robots must recognize human plans and goals from noisy observations of human actions, even when the user acts sub-optimally. Previous works on Plan and Goal Recognition (PGR) as planning have used hierarchical task networks (HTN) to model the actor/human. However, these techniques are insufficient as they do not have user engagement via natural modes of interaction such as language. Moreover, they have no mechanisms to let users, especially those with cognitive impairments, know of a deviation from their original plan or about any sub-optimal actions taken towards their goal. We propose a novel framework for plan and goal recognition in partially observable domains -- Dialogue for Goal Recognition (D4GR) enabling a robot to rectify its belief in human progress by asking clarification questions about noisy sensor data and sub-optimal human actions. We evaluate the performance of D4GR over two simulated domains -- kitchen and blocks domain. With language feedback and the world state information in a hierarchical task model, we show that D4GR framework for the highest sensor noise performs 1% better than HTN in goal accuracy in both domains. For plan accuracy, D4GR outperforms by 4% in the kitchen domain and 2% in the blocks domain in comparison to HTN. The ALWAYS-ASK oracle outperforms our policy by 3% in goal recognition and 7%in plan recognition. D4GR does so by asking 68% fewer questions than an oracle baseline. We also demonstrate a real-world robot scenario in the kitchen domain, validating the improved plan and goal recognition of D4GR in a realistic setting.

Robotics,Artificial Intelligence,Human-Computer Interaction

What problem does this paper attempt to address?

The paper aims to address the problem of effectively recognizing and understanding human intentions during interactions between assistive robots and humans. Specifically, the paper focuses on how to accurately identify the user's plans and goals through noisy observational data when the user (especially those with cognitive impairments) is performing tasks. Current methods such as Hierarchical Task Networks (HTN) can model human behavior but lack the ability to interact with users and cannot provide feedback when users deviate from the original plan or take suboptimal actions. Therefore, the paper proposes a new framework—Dialog-based Goal Recognition (D4GR), which clarifies uncertain information in sensor data and human suboptimal behavior through natural language interaction, thereby improving the accuracy of the robot's understanding of human intentions.

The main contribution of the paper is the combination of HTN's task representation capabilities and the sequential decision-making capabilities of Partially Observable Markov Decision Processes (POMDP), proposing the D4GR framework. This framework can enhance the accuracy of goal recognition and plan recognition through language feedback and environmental state information without frequent inquiries. Experimental results show that in two simulated environments, the kitchen and blocks, D4GR improves goal recognition and plan recognition by 1% and 4% respectively (kitchen environment) compared to traditional HTN methods. Additionally, D4GR has been deployed on a social robot named Kuri, demonstrating its ability to assist users in completing tasks in real-world scenarios.