Go Fetch: Mobile Manipulation in Unstructured Environments

Kenneth Blomqvist,Michel Breyer,Andrei Cramariuc,Julian Förster,Margarita Grinvald,Florian Tschopp,Jen Jen Chung,Lionel Ott,Juan Nieto,Roland Siegwart

DOI: https://doi.org/10.48550/arXiv.2004.00899

2020-04-02

Abstract:With humankind facing new and increasingly large-scale challenges in the medical and domestic spheres, automation of the service sector carries a tremendous potential for improved efficiency, quality, and safety of operations. Mobile robotics can offer solutions with a high degree of mobility and dexterity, however these complex systems require a multitude of heterogeneous components to be carefully integrated into one consistent framework. This work presents a mobile manipulation system that combines perception, localization, navigation, motion planning and grasping skills into one common workflow for fetch and carry applications in unstructured indoor environments. The tight integration across the various modules is experimentally demonstrated on the task of finding a commonly available object in an office environment, grasping it, and delivering it to a desired drop-off location. The accompanying video is available at <a class="link-external link-https" href="https://youtu.be/e89_Xg1sLnY" rel="external noopener nofollow">this https URL</a>.

Robotics,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to enable mobile robots to autonomously perform "search and carry" tasks in unstructured indoor environments. Specifically, the research team has developed a mobile manipulation platform named RoyalYumi, which combines the latest 3D perception, mapping and localization technologies as well as advanced object detection and grasping planning methods, aiming to address the following challenges:

1. **Environment Perception and Navigation**: Robots need to be able to perceive their surroundings in environments not specifically designed for robotic work, plan paths to avoid collisions, and move flexibly to target locations.

2. **Object Detection and Localization**: During the execution of tasks, robots must be able to identify specific target objects and accurately determine their positions. This involves using RGB - D sensors for dense scene reconstruction and leveraging deep - learning methods (such as the YOLOv3 network) for efficient object detection.

3. **Grasping and Manipulation**: Robots also need to have the ability to precisely grasp objects, which requires not only the generation of high - quality grasping pose candidates but also the consideration of kinematic constraints and potential collision risks in actual operations.

4. **High - level Task Planning and Execution**: To simplify task definitions, researchers have designed a series of high - level actions, including searching, approaching, scanning, grasping, and placing, etc. These actions are coordinated and executed through high - level planning frameworks such as state machines or behavior trees.

Through the integration and optimization of the above technologies, this research aims to promote the development of mobile robot technology, enabling it to provide services in more complex and changeable human living environments, and improve operational efficiency, quality, and safety.