Abstract:In Industry 5.0, Digital Twins bring in flexibility and efficiency for smart manufacturing. Recently, the success of artificial intelligence techniques such as deep learning has led to their adoption in manufacturing and especially in human–robot collaboration. Collaborative manufacturing tasks involving human operators and robots pose significant safety and reliability concerns. In response to these concerns, a deep learning-enhanced Digital Twin framework is introduced through which human operators and robots can be detected and their actions can be classified during the manufacturing process, enabling autonomous decision making by the robot control system. Developed using Unreal Engine 4, our Digital Twin framework complies with the Robotics Operating System specification, and supports synchronous control and communication between the Digital Twin and the physical system. In our framework, a fully-supervised detector based on a faster region-based convolutional neural network is firstly trained on synthetic data generated by the Digital Twin, and then tested on the physical system to demonstrate the effectiveness of the proposed Digital Twin-based framework. To ensure safety and reliability, a semi-supervised detector is further designed to bridge the gap between the twin system and the physical system, and improved performance is achieved by the semi-supervised detector compared to the fully-supervised detector that is simply trained on either synthetic data or real data. The evaluation of the framework in multiple scenarios in which human operators collaborate with a Universal Robot 10 shows that it can accurately detect the human and robot, and classify their actions under a variety of conditions. The data from this evaluation have been made publicly available, and can be widely used for research and operational purposes. Additionally, a semi-automated annotation tool from the Digital Twin framework is published to benefit the collaborative robotics community.

Enabling Improved Learning Capability of Industrial Robots with Knowledge Graph Towards Intelligent Digital Twins

Knowledge Graphs in the Digital Twin: A Systematic Literature Review About the Combination of Semantic Technologies and Simulation in Industrial Automation

Cognitive digital twin in manufacturing process: integrating the knowledge graph for enhanced human-centric Industry 5.0

Graph Learning for Cognitive Digital Twins in Manufacturing Systems

Architecture and Implementation of High-Fidelity Digital Twins for Industrial Robots

Scaling Knowledge Graphs for Automating AI of Digital Twins

Digital Twin Graph: Automated Domain-Agnostic Construction, Fusion, and Simulation of IoT-Enabled World

Knowledge-graph-based multi-domain model integration method for digital-twin workshops

Knowledge graph and function block based Digital Twin modeling for robotic machining of large-scale components

Transfer Learning as an Enabler of the Intelligent Digital Twin

A dynamic knowledge graph approach to distributed self-driving laboratories

Digital Twin Meets Knowledge Graph for Intelligent Manufacturing Processes

Embedding Digital Twin Technology in Robotics

A Hybrid Digital Twin Scheme for the Condition Monitoring of Industrial Collaborative Robots

Digital Twin of Industrial Robots Over the Cloud: Framework, Design and Analysis

A deep learning-enhanced Digital Twin framework for improving safety and reliability in human–robot collaborative manufacturing

Digital Twin for Human–Robot Interactions by Means of Industry 4.0 Enabling Technologies

A graph-based knowledge representation and pattern mining supporting the Digital Twin creation of existing manufacturing systems

Digital Twins in Industry 5.0

A Survey on AI-Driven Digital Twins in Industry 4.0: Smart Manufacturing and Advanced Robotics

Digital Twin in Safety-Critical Robotics Applications: Opportunities and Challenges