Abstract:Grasping is critical for intelligent robots to accomplish sophisticated tasks. Even with multimodal sensor fusion, accurately and reliably estimating grasp poses for complex-shaped objects remains a challenge. In this paper, we design a vision-based grasping platform for a more general case, that is, grasping a variety of objects by a simple parallel gripper with the grasp detection model consuming RGB sensing or depth sensing. Focusing on the grasp pose estimation part, we propose a deep grasp detector that uses a densely connected Feature Pyramid Network (FPN) feature extractor and multiple two-stage detection units to achieve dense grasp pose predictions. Specifically, for the feature extractor, the fusion of different layer feature maps can increase both the model's capacity to detect the various size grasp areas and the accuracy of the regressed grasp positions. For each of the two-stage detection unit, the first stage generates horizontal candidate grasp areas, while the second stage refines them to predict the rotated grasp poses. We train and validate our grasp pose estimation algorithm on the Cornell Grasp Dataset and the Jacquard Dataset. The model achieves the detection accuracy of 93.3% and 89.6%, respectively. We further design real-world grasp experiments to verify the effectiveness of our vision-based robotic grasping system. The real scenario trials validate that the system is capable of grasping unseen objects, in particular, achieving robust and accurate grasp pose detection and gripper opening width measurement based on depth sensing only.

A method for robotic grasping based on improved Gaussian mixture model

Towards Feasible Dynamic Grasping: Leveraging Gaussian Process Distance Field, SE(3) Equivariance and Riemannian Mixture Models

A Novel Robotic Grasping Method for Moving Objects Based on Multi-Agent Deep Reinforcement Learning

A Robotic Semantic Grasping Method for Pick-and-place Tasks

Grasping Pose Estimation for Robots Based on Convolutional Neural Networks

Robotic Grasping With Multi-View Image Acquisition and Model-Based Pose Estimation

Towards Precise Model-free Robotic Grasping with Sim-to-Real Transfer Learning

Bayesian Grasp: Robotic visual stable grasp based on prior tactile knowledge

A Vision-based Robot Grasping System

Robotic Grasping Technology Integrating Large Kernel Convolution and Residual Connections

Intelligent Grasping with Natural Human-Robot Interaction.

A grasping posture estimation method based on 3D detection network

A Practical Robotic Grasping Method by Using 6-D Pose Estimation With Protective Correction

A model-free 6-DOF grasp detection method based on point clouds of local sphere area

Robotic Grasp Detection Network Based on Improved Deformable Convolution and Spatial Feature Center Mechanism

Evaluating Gaussian Grasp Maps for Generative Grasping Models

Dynamic Grasp Recognition Using Time Clustering, Gaussian Mixture Models and Hidden Markov Models

A System of Robotic Grasping with Experience Acquisition.

Robotic Grasping Method with 6D Pose Estimation and Point Cloud Fusion

Center-of-Mass-Based Object Regrasping: A Reinforcement Learning Approach and the Effects of Perception Modality

Robotic grasping using visual and tactile sensing