Abstract:Grasping in stacked scenarios is an indispensable capability for intelligent robots. However, in the context of multi-object stacking or occluded scenes, existing algorithms for direct target object grasping result in a high failure rate, while methods for scene clearing grasping lead to inefficiency. Hence, executing grasping operations for target object in a logical sequence is imperative. To address this challenge, we propose an end-to-end grasping model based on a Gated Self Attention Network(GSAN), designed to guide robots to perform optimal sequential grasping of target objects within dense and cluttered scenes. We integrate object detection, grasp detection, and stacking relationship reasoning into a single deep neural network. Specifically, the object detection and grasp detection networks extract features from input RGB images and estimate object categories, bounding boxes and grasp poses. The GSAN captures the non-Euclidean information between object features in high-dimensional space, enhancing the accuracy of triplet relationship reasoning through gated self attention and positional encoding. Our algorithm achieves the best results in the Visual Manipulation Relationship Dataset (VMRD) with an OP of 92.07%, an OR of 91.67%, and an IA of 81.67%, and extensive ablation studies confirm the necessity of each component of our method. As the first end-to-end grasping framework to incorporate self attention into the relationship reasoning module, our proposed method enhances the logical capabilities of robots, enabling efficient grasping operations in complex and dynamic scenes, and fostering human-robot collaboration.

Prioritized Planning for Target-Oriented Manipulation via Hierarchical Stacking Relationship Prediction

Efficient Object Manipulation to an Arbitrary Goal Pose: Learning-based Anytime Prioritized Planning

A Cascaded Deep Learning Framework for Real-time and Robust Grasp Planning

Gated Self Attention Network for Efficient Grasping of Target Objects in Stacked Scenarios

Efficient Stacking and Grasping in Unstructured Environments

Efficient-Learning Grasping and Pushing in Dense Stacking Via Mask Function and Pixel Overlap Rate

An Efficient and Stable Grasp Planning Method for Stacked Parts

Graph-based Visual Manipulation Relationship Reasoning in Object-Stacking Scenes.

Grasp Manipulation Relationship Detection Based on Graph Sample and Aggregation

Robotic Grasping Method Based on 3D Vision for Stacked Rectangular Objects

Robotic Grasping in Multi-Object Stacking Scenes Based on Visual Reasoning

A two-stage grasp detection method for sequential robotic grasping in stacking scenarios

Relationship Detection for Manipulation in Object Stacking Scene with Fully Connected CRF

A graph reasoning method for multi-object unordered stacking scenarios

Robot Grasping Based on Stacked Object Classification Network and Grasping Order Planning

Grasp for Stacking Via Deep Reinforcement Learning

Hierarchical POMDP Planning for Object Manipulation in Clutter

Research on Grasp Planning of Multi-Objects Based on Depth Data

Hierarchical Reinforcement Learning Based on Planning Operators

Task-Oriented Grasping In Object Stacking Scenes With Crf-Based Semantic Model

Visual Manipulation Relationship Recognition in Object-Stacking Scenes