Abstract:Grasping is critical for intelligent robots to accomplish sophisticated tasks. Even with multimodal sensor fusion, accurately and reliably estimating grasp poses for complex-shaped objects remains a challenge. In this paper, we design a vision-based grasping platform for a more general case, that is, grasping a variety of objects by a simple parallel gripper with the grasp detection model consuming RGB sensing or depth sensing. Focusing on the grasp pose estimation part, we propose a deep grasp detector that uses a densely connected Feature Pyramid Network (FPN) feature extractor and multiple two-stage detection units to achieve dense grasp pose predictions. Specifically, for the feature extractor, the fusion of different layer feature maps can increase both the model's capacity to detect the various size grasp areas and the accuracy of the regressed grasp positions. For each of the two-stage detection unit, the first stage generates horizontal candidate grasp areas, while the second stage refines them to predict the rotated grasp poses. We train and validate our grasp pose estimation algorithm on the Cornell Grasp Dataset and the Jacquard Dataset. The model achieves the detection accuracy of 93.3% and 89.6%, respectively. We further design real-world grasp experiments to verify the effectiveness of our vision-based robotic grasping system. The real scenario trials validate that the system is capable of grasping unseen objects, in particular, achieving robust and accurate grasp pose detection and gripper opening width measurement based on depth sensing only.

Guidance - Robot Vision Sensing Platform

Guidance: A visual sensing platform for robotic applications

A Computer Vision for Navigation of Mobile Robots

Robot Guidance Using Machine Vision Techniques in Industrial Environments: A Comparative Review

Active Sensing for Intelligent Robot Vision with Range Imaging Sensor

Application of Three-Dimensional Vision Perception Technology to Industrial Robots

A Vision-based Robot Grasping System

Global Vision System Of Miniature Mobile Robot For Precision Manipulation

Research on hand-eye robot

Quadruped Guidance Robot for the Visually Impaired: A Comfort-Based Approach

HexGuide: A Hexapod Robot for Autonomous Blind Guidance in Challenging Environments.

Vision-Based Real-Time Obstacles Detection and Tracking for Autonomous Vehicle Guidance

Active Sensing and Information Structuring for Intelligent Robot Vision with Range Imaging Sensor

A Visual Guided 4 D.O.F Miniature Robot System for Micro-Handling Purpose

Vision-based Mobile Robot's Environment Outdoor Perception

Robot visual perception and autonomous obstacle avoidance based on deep learning

Design and implementation of an omnidirectional vision system for robot perception

Application of 3D Vision Intelligent Calibration and Imaging Technology for Industrial Robots

A Vision-based Demonstration and Verification Platform for Path Planning

Fast And Accurate Humanoid Robot Navigation Guided By Stereovision

The Optimization of Localization and Navigation for Vision-based Robot