EyeSight Hand: Design of a Fully-Actuated Dexterous Robot Hand with Integrated Vision-Based Tactile Sensors and Compliant Actuation

Branden Romero,Hao-Shu Fang,Pulkit Agrawal,Edward Adelson

2024-08-13

Abstract:In this work, we introduce the EyeSight Hand, a novel 7 degrees of freedom (DoF) humanoid hand featuring integrated vision-based tactile sensors tailored for enhanced whole-hand manipulation. Additionally, we introduce an actuation scheme centered around quasi-direct drive actuation to achieve human-like strength and speed while ensuring robustness for large-scale data collection. We evaluate the EyeSight Hand on three challenging tasks: bottle opening, plasticine cutting, and plate pick and place, which require a blend of complex manipulation, tool use, and precise force application. Imitation learning models trained on these tasks, with a novel vision dropout strategy, showcase the benefits of tactile feedback in enhancing task success rates. Our results reveal that the integration of tactile sensing dramatically improves task performance, underscoring the critical role of tactile information in dexterous manipulation.

Robotics

What problem does this paper attempt to address?

The paper aims to address the following issues:

1. **Design a fully actuated dexterous robotic hand with high-resolution tactile sensors and compliant actuation mechanisms**: The research team designed a novel 7-degree-of-freedom (DoF) humanoid hand named EyeSight Hand. This hand integrates vision-based tactile sensors and employs a quasi-direct drive actuation method to achieve human-like strength and speed while ensuring robustness in large-scale data collection.

2. **Validate the importance of tactile perception in complex manipulation tasks**: Through three challenging tasks (opening a bottle cap, cutting plasticine, and stacking plates), the study demonstrated the significant advantage of tactile feedback in improving task success rates, proving the critical role of tactile information in dexterous manipulation.

3. **Explore imitation learning methods**: A novel visual dropout strategy was proposed to train imitation learning models to perform the aforementioned tasks, thereby better utilizing tactile sensory information and quantitatively demonstrating the performance improvements it brings.

In summary, the core of this research lies in developing a robotic hand that combines advanced tactile sensing technology and efficient actuation schemes, validating its performance in complex environments through practical applications, and further optimizing imitation learning algorithms based on tactile perception.