Abstract:In the context of sustainable manufacturing, efficient collaboration between humans and machines is crucial for improving assembly quality and efficiency. However, traditional methods for action recognition and human–robot collaborative assembly often face challenges such as low efficiency, low accuracy, and poor robustness. To solve such problems, this paper proposes an assembly action-recognition method based on a hybrid convolutional neural network. Firstly, an assembly action-recognition model is proposed using skeletal sequences and a hybrid convolutional neural network model combining Spatial Temporal Graph Convolutional Networks (ST-GCNs) and One-Dimensional Convolutional Neural Networks (1DCNNs) to sense and recognize human behavior actions during the assembly process. This model combines the joint spatial relationship and temporal information extraction ability of the ST-GCN model with the temporal feature extraction ability of the 1DCNN model. By incorporating Batch Normalization (BN) layers and Dropout layers, the generalization performance of the model is enhanced. Secondly, the model is validated on a self-constructed dataset of assembly actions, and the results show that the recognition accuracy of the model can reach 91.7%, demonstrating its superiority. Finally, a digital workshop application system based on digital twins is developed. To test the effectiveness of the proposed method, three sets of control experiments were designed to evaluate both objective and subjective aspects and verify the feasibility of the method presented in this paper. Compared with traditional assembly systems, the proposed method optimizes the recognition of human–robot collaborative assembly actions and applies them to intelligent control systems using digital-twin technology. This intelligent assembly method improves assembly efficiency and saves assembly time. It enables efficient and sustainable collaboration between humans and robots in assembly, leading to a positive and sustainable impact on the manufacturing industry.

Research on Human-Machine Task Collaboration Based on Action Recognition

Action Fusion Recognition Model Based on GAT-GRU Binary Classification Networks for Human-Robot Collaborative Assembly

Hybrid Convolutional Neural Network Approaches for Recognizing Collaborative Actions in Human–Robot Assembly Tasks

Vision-Based Action Recognition Of Construction Workers Using Dense Trajectories

An Attention-Based Deep Learning Approach for Inertial Motion Recognition and Estimation in Human-Robot Collaboration

A Skeleton-Based Assembly Action Recognition Method with Feature Fusion for Human-Robot Collaborative Assembly

Hybrid machine learning for human action recognition and prediction in assembly

Human Action Recognition From Digital Videos Based on Deep Learning.

Human Action Recognition Using Deep Learning Methods.

Data-efficient Multimodal Human Action Recognition for Proactive Human–robot Collaborative Assembly: A Cross-Domain Few-Shot Learning Approach

Ensuring Computers Understand Manual Operations in Production: Deep-Learning-Based Action Recognition in Industrial Workflows

A High-Precision Collaborative Control Algorithm for Multi-Agent System Based on Enhanced Depth Image Fusion Positioning

Deep learning based assembly process action recognition and progress prediction facing human-centric intelligent manufacturing

Vision Sensor Based Action Recognition for Improving Efficiency and Quality under the Environment of Industry 4.0

Action Recognition for Human–Robot Teaming: Exploring Mutual Performance Monitoring Possibilities

A Multimodal Information Fusion Model for Robot Action Recognition with Time Series

Combining Multi-scale Directed Depth Motion Maps and Log-Gabor Filters for Human Action Recognition

An Efficient Human Activity Recognition Framework Based on Graph Convolutional Network for Human-Robot Collaboration

Towards a Safe Human-Robot Collaboration Using Information on Human Worker Activity

Human-Robot Collaboration Through a Multi-Scale Graph Convolution Neural Network With Temporal Attention

Multi-scale Control and Action Recognition Based Human-Robot Collaboration Framework Facing New Generation Intelligent Manufacturing