Semantic Map Based Robot Navigation with Natural Language Input

Xinchi Huang,Guang Yang,Yi Guo
DOI: https://doi.org/10.1109/RO-MAN60168.2024.10731181
2024-08-26
Abstract:We present a new semantic map based robot navigation system in the paper. The system takes human voice input, processes multi-modal data including natural languages and RGB-D images, and generates semantic maps for robot navigation. Making use of recent development in image segmentation tools, we integrate robot mapping and localization with a customized real-time object detection model, so that the semantic and navigation layers are efficiently combined for robot navigation purpose. We demonstrate the performance of our developed algorithms in both simulation and real robot experiments. Compared with existing works, we demonstrate applicability to real robot systems and superior performance in terms of success rate.
Computer Science,Engineering
What problem does this paper attempt to address?