Extracting Dynamic Navigation Goal from Natural Language Dialogue

Lanjun Liang,Ganghui Bian,Huailin Zhao,Yanzhi Dong,Huaping Liu

DOI: https://doi.org/10.1109/iros55552.2023.10342509

2023-01-01

Abstract:Effective access to relevant environmental changes in large human environments is critical for service robots to perform tasks. Since the position of a dynamic goal such as a human is variable, it will be difficult for the robot to locate him accurately. It is worth noting that humans can obtain information through social software, and deal with daily affairs. The current robots search for targets without considering some implicit information changes, which leads to not searching for the target objects in the end. Therefore, we propose to extract human implicit location change information from group chats dialogues, i.e., watching dialogues in group chats and extracting who, when, and where(3W), to assist robots in finding explicit character targets. Then we propose a dynamic spatiotemporal map(DSTM) to store the change information as knowledge for the robot. When the robot identifies a target person, it needs to follow the changing information in the scene to infer the possible location and probability of the target person, and then develop a search strategy. We deployed our framework on a custom mobile robot and performed instruction navigation tasks in a university building to evaluate our approach. We demonstrate the ability of our framework to collect and use information in a large human social environment.