Abstract:Human–robot interaction plays an important role in robots serving human production and life. Object handover between humans and robotics is one of the fundamental problems of human–robot interaction. The majority of current work uses parallel-jaw grippers as the end-effector device, which limits the ability of the robot to grab miscellaneous objects from human and manipulate them subsequently. In this article, we present a framework for human-to-robot dexterous handover using an anthropomorphic hand. The framework takes images captured by two cameras to complete handover scene understanding, grasp configurations prediction, and handover execution. To enable the robot to generalize to diverse delivered objects with miscellaneous shapes and sizes, we propose an anthropomorphic hand grasp network (AHG-Net), an end-to-end network that takes the single-view point clouds of the object as input and predicts the suitable anthropomorphic hand configurations with five different grasp taxonomies. To train our model, we build a large-scale data set with 1M hand grasp annotations from 5K single-view point clouds of 200 objects. We implement a handover system using a UR5 robot arm and HIT-DLR II anthropomorphic robot hand based on our presented framework, which can not only adapt to different human givers but generalize to diverse novel objects with various shapes and sizes. The generalizability, reliability, and robustness of our method are demonstrated on 15 different novel objects with arbitrary handover poses from frontal and lateral positions, a system ablation study, a grasp planner comparison, and a user study on 6 participants delivering 15 objects from two benchmark sets.

Depth Restoration of Hand-Held Transparent Objects for Human-to-Robot Handover

In-Hand 3D Object Reconstruction from a Monocular RGB Video

Stereo Hand-Object Reconstruction for Human-to-Robot Handover

Transparent Object Depth Perception Network for Robotic Manipulation Based on Orientation-aware Guidance and Texture Enhancement

HandFormer: Hand Pose Reconstructing from a Single RGB Image

Visualizing Robot Intent for Object Handovers with Augmented Reality

A4T: Hierarchical Affordance Detection for Transparent Objects Depth Reconstruction and Manipulation

Learning Human-to-Robot Dexterous Handovers for Anthropomorphic Hand

Resolving hand‐object occlusion for mixed reality with joint deep learning and model optimization

Shape and Pose Reconstruction of Robotic In-Hand Objects from a Single Depth Camera

Single Depth View Based Real-Time Reconstruction of Hand-Object Interactions

Reconstructing Hand-Held Objects from Monocular Video.

Fed-HANet: Federated Visual Grasping Learning for Human Robot Handovers

HandNeRF: Learning to Reconstruct Hand-Object Interaction Scene from a Single RGB Image

Human-object Interaction Detection with Depth-Augmented Clues

3D Hand Mesh Recovery from Monocular RGB in Camera Space

ASGrasp: Generalizable Transparent Object Reconstruction and 6-Dof Grasp Detection from RGB-D Active Stereo Camera

Learning Human-to-Robot Handovers from Point Clouds.

ClearDepth: Enhanced Stereo Perception of Transparent Objects for Robotic Manipulation

ASGrasp: Generalizable Transparent Object Reconstruction and Grasping from RGB-D Active Stereo Camera

An Optimized Depth Complementation of Transparent Objects Based Robotic Arm Grasping System