Abstract:Robust and efficient grasping of different objects is still an open problem due to the difficulty of integrating multidisciplinary knowledge such as gripper ontology design, perception, control, and learning. In recent years, learning-based methods have achieved excellent results in grasping various novel objects. However, current methods are usually limited to a single grasping mode or rely on different end effectors to grasp objects of different shapes. For human beings, our hands are capable of grasping various objects with changes in grasping methods and form of hands. In light of this, developing a gripper with similar performance could possibly improve the robot's gripping ability. In this paper, we design a dual-modal soft gripper (DSG) and propose a deep reinforcement learning (DRL) framework to implement the operations. Both of our grasping modes, namely enveloping and pinching, are achieved through the tendon drive system and the deformation of the spring steel plate, which enables the gripper to switch between the two grasping modes in real time. We also combined the cutting-edge achievements of deep learning and reinforcement learning to design an autonomous grasping algorithm based on Q-learning and a deep Q network. Moreover, to fully utilize the visual input from the sensor, we added semantic embeddings of target objects to facilitate the learning, which is especially useful in deciding the grasping method for objects previously unseen. We also evaluate our DRL framework in different scenarios, offering a detailed comparison of each grasping mode and the mixed method (with or without semantic information). Our design has proved efficient in reducing the number of failing grasping actions and improving the success rate when facing novel and tricky objects.

DexRepNet: Learning Dexterous Robotic Grasping Network with Geometric and Spatial Hand-Object Representations

InterRep: A Visual Interaction Representation for Robotic Grasping

DexTransfer: Real World Multi-fingered Dexterous Grasping with Minimal Human Demonstrations

Efficient Residual Learning with Mixture-of-Experts for Universal Dexterous Grasping

Cross-Embodiment Dexterous Grasping with Reinforcement Learning

RealDex: Towards Human-like Grasping for Robotic Dexterous Hand

DexH2R: Task-oriented Dexterous Manipulation from Human to Robots

Vision-Based Robotic Object Grasping—A Deep Reinforcement Learning Approach

Efficient Grasp Detection Network with Gaussian-Based Grasp Representation for Robotic Manipulation

Learning Robust Real-World Dexterous Grasping Policies via Implicit Shape Augmentation

GenDexGrasp: Generalizable Dexterous Grasping

DexPoint: Generalizable Point Cloud Reinforcement Learning for Sim-to-Real Dexterous Manipulation

D(R, O) Grasp: A Unified Representation of Robot and Object Interaction for Cross-Embodiment Dexterous Grasping

Dext-Gen: Dexterous Grasping in Sparse Reward Environments with Full Orientation Control

UniDexGrasp: Universal Robotic Dexterous Grasping via Learning Diverse Proposal Generation and Goal-Conditioned Policy

Dexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Cost

DexDiff: Towards Extrinsic Dexterity Manipulation of Ungraspable Objects in Unrestricted Environments

Implementation and Optimization of Grasping Learning with Dual-modal Soft Gripper.

DexVIP: Learning Dexterous Grasping with Human Hand Pose Priors from Video

Research on Perception and Control Technology for Robot Dexterous Operation

Dexterous Functional Grasping