Intelligent LiDAR Navigation: Leveraging External Information and Semantic Maps with LLM as Copilot

Fujing Xie,Jiajie Zhang,Sören Schwertfeger
2024-09-13
Abstract:Traditional robot navigation systems primarily utilize occupancy grid maps and laser-based sensing technologies, as demonstrated by the popular move_base package in ROS. Unlike robots, humans navigate not only through spatial awareness and physical distances but also by integrating external information, such as elevator maintenance updates from public notification boards and experiential knowledge, like the need for special access through certain doors. With the development of Large Language Models (LLMs), which posses text understanding and intelligence close to human performance, there is now an opportunity to infuse robot navigation systems with a level of understanding akin to human cognition. In this study, we propose using osmAG (Area Graph in OpensStreetMap textual format), an innovative semantic topometric hierarchical map representation, to bridge the gap between the capabilities of ROS move_base and the contextual understanding offered by LLMs. Our methodology employs LLMs as actual copilot in robot navigation, enabling the integration of a broader range of informational inputs while maintaining the robustness of traditional robotic navigation systems. Our code, demo, map, experiment results can be accessed at <a class="link-external link-https" href="https://github.com/xiexiexiaoxiexie/Intelligent-LiDAR-Navigation-LLM-as-Copilot" rel="external noopener nofollow">this https URL</a>.
Robotics
What problem does this paper attempt to address?
The paper attempts to address the problem of how robot navigation systems can better adapt to environmental changes in large dynamic environments, particularly when encountering unexpected obstacles, by utilizing external information (such as information on public notice boards) for path planning to improve the robustness and adaptability of navigation. Specifically, traditional robot navigation systems mainly rely on occupancy grid maps and laser sensing technology, which have limitations in large-scale dynamic scenarios with frequent environmental changes. For example, a delivery robot on a campus might be unable to complete its task due to an unforeseen road closure caused by pipeline maintenance, even if this information has been posted on a public website, the robot cannot access it. To solve this problem, the authors propose a new method that uses large language models (LLMs) as a co-pilot for the navigation system, combined with osmAG (a semantic topological map representation based on OpenStreetMap text format), enabling the robot to understand and utilize external information, such as public notices and weather, to achieve more intelligent and adaptive navigation. This method not only improves the robot's responsiveness to environmental changes but also maintains the robustness of traditional robot navigation systems, ensuring efficient navigation in complex indoor environments.