Abstract:Human-robot collaborative (HRC) assembly has become popular in recent years. It takes full advantage of the strength, repeatability and accuracy of robots and the high-level cognition, flexibility and adaptability of humans to achieve an ergonomic working environment with better overall productivity. However, HRC assembly is still in its infancy nowadays. How to ensure the safety and efficiency of HRC assembly while reducing assembly failures caused by human errors is challenging. To address the current challenges, this paper proposes a novel human-cyber-physical assembly system (HCPaS) framework, which combines the powerful perception and control capacity of digital twin with the virtual-reality interaction capacity of augmented reality (AR) to achieve a safe and efficient HRC environment. Based on the framework, a deep learning-enabled fusion method of HCPaS is proposed from the perspective of robot-level fusion and part-level fusion. Robot-level fusion perceives the pose of robots with the combination of PointNet and iterative closest point (ICP) algorithm, where the status of robots together with their surroundings could be registered into AR environment to improve the human's cognitive ability of complex assembly environment, thus ensuring the safe HRC assembly. Part-level fusion recognizes the type and pose of parts being assembled with a parallel network that takes an extended Pixel-wise Voting Network (PVNet) as the base architecture, on which assembly sequence/process information of the part could be registered into AR environment to provide smart guidance for manual work to avoid human errors. Eventually, experimental results demonstrate the effectiveness and efficiency of the approach.

Proactive Human-Robot Co-Assembly: Leveraging Human Intention Prediction and Robust Safe Control

A Cooperative Shared Control Scheme Based on Intention Recognition for Flexible Assembly Manufacturing

Proactive Human–robot Collaboration: Mutual-cognitive, Predictable, and Self-Organising Perspectives

Toward Proactive Human–Robot Collaborative Assembly: A Multimodal Transfer-Learning-Enabled Action Prediction Approach

Towards Efficient Human-Robot Collaboration With Robust Plan Recognition and Trajectory Prediction

Safety-aware human-centric collaborative assembly

Proactive Robot Control for Collaborative Manipulation Using Human Intent

A reinforcement learning method for human-robot collaboration in assembly tasks

A Deep Learning-Enabled Human-Cyber-physical Fusion Method Towards Human-Robot Collaborative Assembly.

Towards Better Human Robot Collaboration with Robust Plan Recognition and Trajectory Prediction

“Follower” to “Collaborator”: A robot proactive collaborative controller based on human multimodal information for 3D handling/assembly scenarios

Robust Task Planning for Assembly Lines with Human-Robot Collaboration

Human-in-the-loop Robotic Manipulation Planning for Collaborative Assembly

A collaborative intelligence-based approach for handling human-robot collaboration uncertainties

Overview of Human-Robot Collaboration in Manufacturing

Human-robot collaboration assembly personalised safety detection method based on digital twin

Human-robot Collaborative Assembly and Welding: A Review and Analysis of the State of the Art

HRC for dual-robot intelligent assembly system based on multimodal perception

How to be Helpful? Supportive Behaviors and Personalization for Human-Robot Collaboration

Human–Robot Collaboration in Smart Manufacturing: Robot Reactive Behavior Intelligence

Human–Robot Collaboration Framework Based on Impedance Control in Robotic Assembly