Abstract:During the execution of handling processes in manufacturing, it is difficult to measure the process forces with state-of-the-art gripper systems since they usually lack integrated sensors. Thus, the exact state of the gripped object and the actuating process forces during manipulation and handling are unknown. This paper proposes a deep learning regression model to construct a continuous stability metric to predict the maximum process forces on the gripped objects using high-resolution optical tactile sensors. A pull experiment was developed to obtain a valid dataset for training. Continuously force-based labeled pairs of tactile images for varying grip positions of industrial gearbox parts were acquired to train a novel neural network inspired by encoder-decoder architectures. A ResNet-18 model was used for comparison. Both models can predict the maximum process force for each object with a precision of less than 1 N. During validation, the generalization potential of the proposed methodology with respect to previously unknown objects was demonstrated with an accuracy of 0.4-2.1 N and precision of 1.7-3.4 N, respectively.

What problem does this paper attempt to address?

This paper aims to address the problem of predicting the maximum allowable process force for grasped objects during industrial assembly processes. Specifically, the research focuses on the following aspects: 1. **Problem Background**: In manufacturing, traditional gripping systems often lack integrated sensors when performing handling and assembly tasks, making it difficult to accurately measure process forces. This results in the state of the grasped object and the forces acting on it during handling and operation being unknown. 2. **Research Objective**: Propose a method based on a deep learning regression model to construct continuous stability indicators to predict the maximum process force on the grasped object. This prediction is crucial for determining whether to continue the task or re-grasp, as failed attempts can result in time and money losses, disrupt the production line, and even damage parts or robots. 3. **Methodology**: - Developed an experimental scheme to obtain an effective dataset, including tactile images recorded by high-resolution optical tactile sensors at different grasping positions and corresponding continuous force labels. - Designed a novel neural network (inspired by the encoder-decoder architecture) and compared it with the ResNet-18 model. These models can predict the maximum process force for each object based on tactile images, with an accuracy of less than 1 Newton. - Demonstrated the generalization potential of the method for previously unknown objects during the validation phase, with an accuracy of 0.4 to 2.1 Newtons and a precision of 1.7 to 3.4 Newtons. 4. **Experimental Design**: The experiments used a UR5e robot, a WSG-50 parallel gripper, and two GelSight R1.5 optical tactile sensors. By applying static force to the grasped object, the maximum force value at which it begins to slip from the grasping point was measured. The experiments also considered different types of industrial gear components and conducted data collection and model training. 5. **Main Contributions**: Compared to existing methods, the proposed method uses force measurements to generate continuous labels rather than being limited to success probabilities. Additionally, a new neural network structure is introduced to convert two input images into force values, adopting an encoder-decoder structure. In summary, the main purpose of this paper is to develop a method capable of accurately predicting the maximum allowable process force on grasped objects to improve the success rate and efficiency of industrial assembly tasks.

Predicting Maximum Permitted Process Forces for Object Grasping and Manipulation Using a Deep Learning Regression Model

Active Learning-Based Grasp for Accurate Industrial Manipulation.

Arm Movements Effect On Grasping Force Prediction Using Surface Electromyography Signals

A Deep Learning Method for Vision Based Force Prediction of a Soft Fin Ray Gripper Using Simulation Data

Learning to Predict Grip Quality from Simulation: Establishing a Digital Twin to Generate Simulated Data for a Grip Stability Metric

Force-Guided High-Precision Grasping Control of Fragile and Deformable Objects Using sEMG-Based Force Prediction

Spatiotemporal modeling of grip forces captures proficiency in manual robot control

Towards Creating a Deployable Grasp Type Probability Estimator for a Prosthetic Hand

Gaining a Sense of Touch. Physical Parameters Estimation using a Soft Gripper and Neural Networks

Estimation and early prediction of grip force based on sEMG signals and deep recurrent neural networks

Estimation Of Hand Grasp Force Based On Forearm Surface Emg

Learning Gentle Grasping from Human-Free Force Control Demonstration

Analytic and Data-Driven Force Prediction for Vacuum-Based Granular Grippers

Deep Learning Method for Grasping Novel Objects Using Dexterous Hands

Joint angle prediction for a cable-driven gripper with variable joint stiffness through numerical modeling and machine learning

Predict Robot Grasp Outcomes Based On Multi-Modal Information

Learning to Detect Slip through Tactile Estimation of the Contact Force Field and its Entropy

Deep Neural Network Approach in EMG-Based Force Estimation for Human–Robot Interaction

Robust Learning of Tactile Force Estimation through Robot Interaction

Learning to Predict Friction and Classify Contact States by Tactile Sensor

Dex-Net 3.0: Computing Robust Robot Vacuum Suction Grasp Targets in Point Clouds using a New Analytic Model and Deep Learning