Visual Pressure Estimation and Control for Soft Robotic Grippers

Patrick Grady,Jeremy A. Collins,Samarth Brahmbhatt,Christopher D. Twigg,Chengcheng Tang,James Hays,Charles C. Kemp

DOI: https://doi.org/10.48550/arXiv.2204.07268

2022-08-10

Abstract:Soft robotic grippers facilitate contact-rich manipulation, including robust grasping of varied objects. Yet the beneficial compliance of a soft gripper also results in significant deformation that can make precision manipulation challenging. We present visual pressure estimation & control (VPEC), a method that infers pressure applied by a soft gripper using an RGB image from an external camera. We provide results for visual pressure inference when a pneumatic gripper and a tendon-actuated gripper make contact with a flat surface. We also show that VPEC enables precision manipulation via closed-loop control of inferred pressure images. In our evaluation, a mobile manipulator (Stretch RE1 from Hello Robot) uses visual servoing to make contact at a desired pressure; follow a spatial pressure trajectory; and grasp small low-profile objects, including a microSD card, a penny, and a pill. Overall, our results show that visual estimates of applied pressure can enable a soft gripper to perform precision manipulation.

Robotics

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: how to estimate the pressure exerted by soft robotic grippers through visual methods, and use these pressure estimates to achieve precise grasping and manipulation tasks. Specifically, the researchers proposed a method named VPEC (Visual Pressure Estimation & Control), aiming to estimate the pressure distribution exerted when the soft robotic gripper contacts a plane from a single RGB image captured by an external camera, and achieve precise control of the target pressure through closed - loop control.

### Core Problems of the Paper
1. **Challenges Brought by the High Flexibility of Soft Robotic Grippers**:
- Due to their high flexibility, soft robotic grippers will deform significantly when in contact with objects, which makes precise control and manipulation difficult.
- Traditional rigid - body models cannot accurately describe the mechanical behavior of soft grippers, and embedded sensors increase hardware complexity and may change the mechanical characteristics of the grippers.

2. **The Need for Visual Estimation of Pressure**:
- The researchers hope to directly estimate the pressure exerted by the grippers through visual methods, thereby avoiding complex modeling and sensor calibration problems.
- Visual estimation can utilize information such as visible deformation, shadow, and motion blur of the grippers, providing a simple and effective method for pressure estimation.

### Solutions
- **VPEC Method**:
- Use a convolutional neural network (VPEC - Net) to estimate the pressure distribution from a single RGB image.
- The pressure estimation results are used for closed - loop control, enabling the robot to precisely control the exerted pressure and complete fine manipulation tasks such as pressing buttons, flipping switches, or grasping small objects.

- **Experimental Verification**:
- The researchers conducted experiments using two different types of soft grippers (tendon - driven grippers and pneumatic grippers) to verify the effectiveness of the VPEC method.
- The experimental results show that VPEC can achieve high - precision pressure control and grasping operations in multiple tasks, including grasping small low - profile objects such as micro - SD cards and coins.

### Main Contributions
1. **Proposed the VPEC Algorithm**: Estimate the pressure exerted by soft grippers through a single RGB image.
2. **Achieved Precise Manipulation**: Achieve the pressure target through closed - loop control and successfully grasp a variety of small objects.
3. **Released Datasets and Code**: Provide training datasets, models, and code to support subsequent research and reproducibility.

### Conclusion
This paper demonstrates the application potential of visual pressure estimation in soft robots. By estimating pressure through visual methods and performing closed - loop control, it solves the problem of precise manipulation of soft grippers under high flexibility.