Abstract:We propose an active learning-based grasp method for accurate industrial manipulation that combines the high accuracy of geometrically driven grasp methods and the generalization ability of data-driven grasp methods. Our grasp sequence consists of pregrasp stage and grasp stage which integrates the active perception and manipulation. In pregrasp stage, the manipulator actively moves and perceives the object. At each step, given the perception image, a motion is chosen so that the manipulator can adjust to a proper pose to grasp the object. We train a convolutional neural network to estimate the motion and combine the network with a closed-loop control so that the end effector can move to the pregrasp state. In grasp stage, the manipulator executes a fixed motion to complete the grasp task. The fixed motion can be acquired from the demonstration with nonexpert conveniently. Our proposed method does not require the prior knowledge of camera intrinsic parameters, hand-eye transformation, or manually designed feature of objects. Instead, the training data sets containing prior knowledge are collected through interactive perception. The method can be easily transferred to new tasks with a few human interventions and is able to complete high accuracy grasp task with a certain robustness to partial observation condition. In our circuit board grasping tests, we could achieve a grasp accuracy of 0.8 mm and 0.6°. Note to Practitioners—The research in this paper is motivated by the following practical problem. Manipulators on industrial lines can complete high accuracy tasks with hand-crafting features of objects. The perception is only used for object detection and localization. It is not flexible since the prior knowledge differs from tasks, which takes a long time to deploy in a new task. Besides, only well-trained experts are qualified to complete the deployment process. Our grasp method uses a convolutional network to estimate the motion for manipulator directly from images. The camera is mounted on the manipulator and can perceive the object actively. The training data set of the network is specific for different objects that can be automatically collected with a few human interventions. Our method simplifies the deployment process and can be applied in 3C industry (computers, communications, and consumer electronics) where the products upgrade frequently.

Leveraging Foundation Model Automatic Data Augmentation Strategies and Skeletal Points for Hands Action Recognition in Industrial Assembly Lines

Learning-Based Hand Motion Capture and Understanding in Assembly Process

Active Learning-Based Grasp for Accurate Industrial Manipulation.

A Novel Multi-Stream Hand-Object Interaction Network for Assembly Action Recognition

A Skeleton-Based Assembly Action Recognition Method with Feature Fusion for Human-Robot Collaborative Assembly

An Attention-Based Deep Learning Approach for Inertial Motion Recognition and Estimation in Human-Robot Collaboration

Proving the Potential of Skeleton Based Action Recognition to Automate the Analysis of Manual Processes

Ensuring Computers Understand Manual Operations in Production: Deep-Learning-Based Action Recognition in Industrial Workflows

Low-Cost and Real-Time Industrial Human Action Recognitions Based on Large-Scale Foundation Models

A flexible manufacturing assembly system with deep reinforcement learning

Hand Gesture Recognition of Methods-Time Measurement-1 Motions in Manual Assembly Tasks Using Graph Convolutional Networks

Deep learning based assembly process action recognition and progress prediction facing human-centric intelligent manufacturing

Augmented Reality Assisted Assembly Training Oriented Dynamic Gesture Recognition and Prediction

Self-supervised Optimization of Hand Pose Estimation using Anatomical Features and Iterative Learning

A Human–Robot Collaboration Method Using a Pose Estimation Network for Robot Learning of Assembly Manipulation Trajectories From Demonstration Videos

Skeleton-based Action Recognition for Industrial Packing Process

Hybrid Convolutional Neural Network Approaches for Recognizing Collaborative Actions in Human–Robot Assembly Tasks

Automated assembly quality inspection by deep learning with 2D and 3D synthetic CAD data

Data-efficient Multimodal Human Action Recognition for Proactive Human–robot Collaborative Assembly: A Cross-Domain Few-Shot Learning Approach

Learning Human-to-Robot Dexterous Handovers for Anthropomorphic Hand

Adaptive Real-Time Similar Repetitive Manual Procedure Prediction and Robotic Procedure Generation for Human-Robot Collaboration.