Abstract:We propose an active learning-based grasp method for accurate industrial manipulation that combines the high accuracy of geometrically driven grasp methods and the generalization ability of data-driven grasp methods. Our grasp sequence consists of pregrasp stage and grasp stage which integrates the active perception and manipulation. In pregrasp stage, the manipulator actively moves and perceives the object. At each step, given the perception image, a motion is chosen so that the manipulator can adjust to a proper pose to grasp the object. We train a convolutional neural network to estimate the motion and combine the network with a closed-loop control so that the end effector can move to the pregrasp state. In grasp stage, the manipulator executes a fixed motion to complete the grasp task. The fixed motion can be acquired from the demonstration with nonexpert conveniently. Our proposed method does not require the prior knowledge of camera intrinsic parameters, hand-eye transformation, or manually designed feature of objects. Instead, the training data sets containing prior knowledge are collected through interactive perception. The method can be easily transferred to new tasks with a few human interventions and is able to complete high accuracy grasp task with a certain robustness to partial observation condition. In our circuit board grasping tests, we could achieve a grasp accuracy of 0.8 mm and 0.6°. Note to Practitioners—The research in this paper is motivated by the following practical problem. Manipulators on industrial lines can complete high accuracy tasks with hand-crafting features of objects. The perception is only used for object detection and localization. It is not flexible since the prior knowledge differs from tasks, which takes a long time to deploy in a new task. Besides, only well-trained experts are qualified to complete the deployment process. Our grasp method uses a convolutional network to estimate the motion for manipulator directly from images. The camera is mounted on the manipulator and can perceive the object actively. The training data set of the network is specific for different objects that can be automatically collected with a few human interventions. Our method simplifies the deployment process and can be applied in 3C industry (computers, communications, and consumer electronics) where the products upgrade frequently.

Estimating Pose of Object and Manipulator Grasping Control

Robust Kinematic Calibration for Improving Collaboration Accuracy of Dual-Arm Manipulators with Experimental Validation

Robust And Accurate Multiple-Camera Pose Estimation Toward Robotic Applications

6D Pose Estimation Method of Metal Parts for Robotic Grasping Based on Semantic-Level Line Matching

Active Learning-Based Grasp for Accurate Industrial Manipulation.

A Practical Robotic Grasping Method by Using 6-D Pose Estimation With Protective Correction

Six-dimensional Target Pose Estimation for Robot Autonomous Manipulation: Methodology and Verification

A Self-Supervised Learning Manipulator Grasping Approach Based on Instance Segmentation.

Single-Camera Multi-View 6DoF pose estimation for robotic grasping

Kinect-based Object Grasping by Manipulator

3D Foundation Models Enable Simultaneous Geometry and Pose Estimation of Grasped Objects

Robotic Grasping With Multi-View Image Acquisition and Model-Based Pose Estimation

A Robotic Semantic Grasping Method for Pick-and-place Tasks

6D Pose Estimation with Combined Deep Learning and 3D Vision Techniques for a Fast and Accurate Object Grasping

Vision-Based Categorical Object Pose Estimation and Manipulation.

6-D Object Pose Estimation Based on Point Pair Matching for Robotic Grasp Detection

Instance-level 6D pose estimation based on multi-task parameter sharing for robotic grasping

Unified Motion/Force/Impedance Control for Manipulators in Unknown Contact Environments Based on Robust Model-Reaching Approach

Physics-Based Object 6D-Pose Estimation during Non-Prehensile Manipulation

A Pose Estimation and Optimization Method for Mobile Manipulator’s End-effectors Based on Stereo Vision and ICoP Algorithm*

RGBManip: Monocular Image-based Robotic Manipulation through Active Object Pose Estimation