AvTF: A Visual-Tactile Fingertip with Active Sensing and Manipulation

Jie Zhao,Sicheng Wang,Jianhua Shan,Yuhao Sun,Shixin Zhang,Fuchun Sun,Bin Fang
DOI: https://doi.org/10.1109/wrcsara64167.2024.10685798
2024-01-01
Abstract:In the interaction between dexterous hands and the environment, perception and operation are inseparable. Perception provides information and feedback for operations, and operations in turn affect the perception. Current VTS(visual-tactile sensors) lack initiative and cannot sense the state of the operated object in the world coordinate system. The AvTF(Active visual-tactile Fingertip) introduced in this article is a vision-based fingertip (finger-shaped full-field sensing) tactile sensor that uses a camera to capture tactile images. Sensing and operating are integrated by adding active DoF (Degree of Freedom). Camera modules under different coordinate systems can provide various perspectives for sensors. Through a series of experimental verifications, AvTF has achieved real-time feedback on the grasping and manipulation of objects in the hand, providing a new solution for dexterous hands to fuse sensing and operation better.
What problem does this paper attempt to address?