Abstract:The 6-Degree-of-Freedom (6-DoF) robotic grasping is a fundamental task in robot manipulation, aimed at detecting graspable points and corresponding parameters in a 3D space, i.e affordance learning, and then a robot executes grasp actions with the detected affordances. Existing research works on affordance learning predominantly focus on learning local features directly for each grid in a voxel scene or each point in a point cloud scene, subsequently filtering the most promising candidate for execution. Contrarily, cognitive models of grasping highlight the significance of global descriptors, such as size, shape, and orientation, in grasping. These global descriptors indicate a grasp path closely tied to actions. Inspired by this, we propose a novel bio-inspired neural network that explicitly incorporates global feature encoding. In particular, our method utilizes a Truncated Signed Distance Function (TSDF) as input, and employs the recently proposed Transformer model to encode the global features of a scene directly. With the effective global representation, we then use deconvolution modules to decode multiple local features to generate graspable candidates. In addition, to integrate global and local features, we propose using a skip-connection module to merge lower-layer global features with higher-layer local features. Our approach, when tested on a recently proposed pile and packed grasping dataset for a decluttering task, surpassed state-of-the-art local feature learning methods by approximately 5% in terms of success and declutter rates. We also evaluated its running time and generalization ability, further demonstrating its superiority. We deployed our model on a Franka Panda robot arm, with real-world results aligning well with simulation data. This underscores our approach's effectiveness for generalization and real-world applications.

Learning Sim-to-Real Dense Object Descriptors for Robotic Manipulation

Learning Robot Manipulation Skills from Human Demonstration Videos Using Two-Stream 2-D/3-D Residual Networks with Self-Attention

DexRepNet: Learning Dexterous Robotic Grasping Network with Geometric and Spatial Hand-Object Representations

Dense-Descriptor Transporter Networks for Multidimensional Deformable Object Manipulation

Vision-Based Robotic Object Grasping—A Deep Reinforcement Learning Approach

SimNet: Enabling Robust Unknown Object Manipulation from Pure Synthetic Data via Stereo

Vision-Based Categorical Object Pose Estimation and Manipulation.

Robotic picking in dense clutter via domain invariant learning from synthetic dense cluttered rendering

Dynamics Learning with Object-Centric Interaction Networks for Robot Manipulation

Learning Latent Object-Centric Representations for Visual-Based Robot Manipulation

Deep Learning-Based Ensemble Approach for Autonomous Object Manipulation with an Anthropomorphic Soft Robot Hand

Toward Zero-Shot Sim-to-Real Transfer Learning for Pneumatic Soft Robot 3D Proprioceptive Sensing

Bio-inspired affordance learning for 6-DoF robotic grasping: A transformer-based global feature encoding approach

Towards Precise Model-free Robotic Grasping with Sim-to-Real Transfer Learning

Robotic Continuous Grasping System by Shape Transformer-Guided Multi-Object Category-Level 6D Pose Estimation

Autonomous Robotic Manipulation: Real-Time, Deep-Learning Approach for Grasping of Unknown Objects

Learning 6-DoF Object Poses to Grasp Category-level Objects by Language Instructions

Category-level Object Detection, Pose Estimation and Reconstruction from Stereo Images

CORN: Contact-based Object Representation for Nonprehensile Manipulation of General Unseen Objects

Semantic Segmentation and 6DoF Pose Estimation using RGB-D Images and Deep Neural Networks

Visual Servoing with Deep Learning and Data Augmentation for Robotic Manipulation