Autonomous Search of Semantic Objects in Unknown Environments

Zhentian Qian,Jie Fu,Jing Xiao

2023-11-21

Abstract:This paper addresses the problem of enabling a robot to search for a semantic object, i.e., an object with a semantic label, in an unknown and GPS-denied environment. For the robot in the unknown environment to detect and find the target semantic object, it must perform simultaneous localization and mapping (SLAM) at both geometric and semantic levels using its onboard sensors while planning and executing its motion based on the ever-updated SLAM results. In other words, the robot must be able to conduct simultaneous localization, semantic mapping, motion planning, and execution in real-time in the presence of sensing and motion uncertainty. This is an open problem as it combines semantic SLAM based on perception and real-time motion planning and execution under uncertainty. Moreover, the goals of the robot motion change on the fly depending on whether and how the robot can detect the target object. We propose a novel approach to tackle the problem, leveraging semantic SLAM, Bayesian Networks, Markov Decision Process, and Real-Time Dynamic Programming. The results in simulation and real experiments demonstrate the effectiveness and efficiency of our approach.

Robotics

What problem does this paper attempt to address?

The paper attempts to address the problem of enabling a robot to autonomously search for target objects with semantic labels in an unknown environment without GPS signals. Specifically, the robot needs to detect and find the target objects in such an environment while performing simultaneous localization and mapping (SLAM) at both geometric and semantic levels. Based on the continuously updated SLAM results, the robot must plan and execute its movements. This involves achieving real-time localization, semantic mapping, motion planning, and execution under perception and motion uncertainties. It is an open problem because it combines perception-based semantic SLAM with real-time motion planning and execution under uncertainty.

The paper points out that the robot's motion goals will dynamically change based on whether and how it detects the target objects. To solve this problem, the authors propose a new method that utilizes semantic SLAM, Bayesian networks, Markov decision processes (MDP), and real-time dynamic programming (RTDP). Experimental results show that this method is effective and efficient in both simulated and real-world experiments.