Force-EvT: A Closer Look at Robotic Gripper Force Measurement with Event-based Vision Transformer

Qianyu Guo,Ziqing Yu,Jiaming Fu,Yawen Lu,Yahya Zweiri,Dongming Gan

2024-04-01

Abstract:Robotic grippers are receiving increasing attention in various industries as essential components of robots for interacting and manipulating objects. While significant progress has been made in the past, conventional rigid grippers still have limitations in handling irregular objects and can damage fragile objects. We have shown that soft grippers offer deformability to adapt to a variety of object shapes and maximize object protection. At the same time, dynamic vision sensors (e.g., event-based cameras) are capable of capturing small changes in brightness and streaming them asynchronously as events, unlike RGB cameras, which do not perform well in low-light and fast-moving environments. In this paper, a dynamic-vision-based algorithm is proposed to measure the force applied to the gripper. In particular, we first set up a DVXplorer Lite series event camera to capture twenty-five sets of event data. Second, motivated by the impressive performance of the Vision Transformer (ViT) algorithm in dense image prediction tasks, we propose a new approach that demonstrates the potential for real-time force estimation and meets the requirements of real-world scenarios. We extensively evaluate the proposed algorithm on a wide range of scenarios and settings, and show that it consistently outperforms recent approaches.

Robotics,Image and Video Processing

What problem does this paper attempt to address?

The paper aims to address the issue of force measurement in soft robotic grippers during operation, particularly focusing on the limitations of traditional rigid grippers when handling irregular or fragile objects. The authors propose a novel method based on dynamic vision sensors (event cameras) and Vision Transformers (ViT) to estimate the force applied to the soft robotic gripper.

Specifically, the main contributions of the paper include:
1. **Proposing a new force measurement method**: Utilizing dynamic vision sensors to capture event data during the deformation process of the robotic gripper and estimating the force through a Vision Transformer network.
2. **Constructing a dataset**: Named RG-Event, the dataset contains 1000 event frames and their corresponding force labels, used for training and validating the proposed algorithm.
3. **Adopting Vision Transformers as the core architecture**: Demonstrating the good performance of Vision Transformers in regression tasks, capable of real-time force estimation and meeting the needs of practical application scenarios.

The paper details the experimental setup, data collection process, and the specific method of using Vision Transformers for force measurement. Experimental results show that this method significantly improves the accuracy of force measurement compared to previous methods. Additionally, the authors discuss future research directions, such as testing system performance under different lighting conditions and extending to more complex gripper designs.