Abstract:We propose an active learning-based grasp method for accurate industrial manipulation that combines the high accuracy of geometrically driven grasp methods and the generalization ability of data-driven grasp methods. Our grasp sequence consists of pregrasp stage and grasp stage which integrates the active perception and manipulation. In pregrasp stage, the manipulator actively moves and perceives the object. At each step, given the perception image, a motion is chosen so that the manipulator can adjust to a proper pose to grasp the object. We train a convolutional neural network to estimate the motion and combine the network with a closed-loop control so that the end effector can move to the pregrasp state. In grasp stage, the manipulator executes a fixed motion to complete the grasp task. The fixed motion can be acquired from the demonstration with nonexpert conveniently. Our proposed method does not require the prior knowledge of camera intrinsic parameters, hand-eye transformation, or manually designed feature of objects. Instead, the training data sets containing prior knowledge are collected through interactive perception. The method can be easily transferred to new tasks with a few human interventions and is able to complete high accuracy grasp task with a certain robustness to partial observation condition. In our circuit board grasping tests, we could achieve a grasp accuracy of 0.8 mm and 0.6°. Note to Practitioners—The research in this paper is motivated by the following practical problem. Manipulators on industrial lines can complete high accuracy tasks with hand-crafting features of objects. The perception is only used for object detection and localization. It is not flexible since the prior knowledge differs from tasks, which takes a long time to deploy in a new task. Besides, only well-trained experts are qualified to complete the deployment process. Our grasp method uses a convolutional network to estimate the motion for manipulator directly from images. The camera is mounted on the manipulator and can perceive the object actively. The training data set of the network is specific for different objects that can be automatically collected with a few human interventions. Our method simplifies the deployment process and can be applied in 3C industry (computers, communications, and consumer electronics) where the products upgrade frequently.

Initial Analysis of Data-Driven Haptic Search for the Smart Suction Cup

Haptic search with the Smart Suction Cup on adversarial objects

Dexterous Manoeuvre Through Touch in a Cluttered Scene

Active Learning-Based Grasp for Accurate Industrial Manipulation.

Realtime State Estimation with Tactile and Visual Sensing for Inserting a Suction-held Object

Dex-Net 3.0: Computing Robust Robot Vacuum Suction Grasp Targets in Point Clouds using a New Analytic Model and Deep Learning

Learning to Detect Slip through Tactile Estimation of the Contact Force Field and its Entropy

Leveraging distributed contact force measurements for slip detection: a physics-based approach enabled by a data-driven tactile sensor

Learning to Predict Grip Quality from Simulation: Establishing a Digital Twin to Generate Simulated Data for a Grip Stability Metric

Towards Haptic-Based Dual-Arm Manipulation

Design and Experimental Evaluation of a Sensorimotor-Inspired Grasping Strategy for Dexterous Prosthetic Hands

Grasp Failure Constraints for Fast and Reliable Pick-and-Place Using Multi-Suction-Cup Grippers

Model-free Grasping with Multi-Suction Cup Grippers for Robotic Bin Picking

Learning Haptic-Based Object Pose Estimation for In-Hand Manipulation Control With Underactuated Robotic Hands

More Than a Feeling: Learning to Grasp and Regrasp using Vision and Touch

Acquisition and Prediction of High-Density Tactile Field Data for Rigid and Flexible Objects

Non-Parametric Self-Identification and Model Predictive Control of Dexterous In-Hand Manipulation

Robust Learning-Based Incipient Slip Detection using the PapillArray Optical Tactile Sensor for Improved Robotic Gripping

Digital Twin-Enabled Grasp Outcomes Assessment for Unknown Objects Using Visual-Tactile Fusion Perception

AViTa: Adaptive Visual-Tactile Dexterous Grasping

Analysing the Interplay of Vision and Touch for Dexterous Insertion Tasks