Abstract:In the context of sustainable manufacturing, efficient collaboration between humans and machines is crucial for improving assembly quality and efficiency. However, traditional methods for action recognition and human–robot collaborative assembly often face challenges such as low efficiency, low accuracy, and poor robustness. To solve such problems, this paper proposes an assembly action-recognition method based on a hybrid convolutional neural network. Firstly, an assembly action-recognition model is proposed using skeletal sequences and a hybrid convolutional neural network model combining Spatial Temporal Graph Convolutional Networks (ST-GCNs) and One-Dimensional Convolutional Neural Networks (1DCNNs) to sense and recognize human behavior actions during the assembly process. This model combines the joint spatial relationship and temporal information extraction ability of the ST-GCN model with the temporal feature extraction ability of the 1DCNN model. By incorporating Batch Normalization (BN) layers and Dropout layers, the generalization performance of the model is enhanced. Secondly, the model is validated on a self-constructed dataset of assembly actions, and the results show that the recognition accuracy of the model can reach 91.7%, demonstrating its superiority. Finally, a digital workshop application system based on digital twins is developed. To test the effectiveness of the proposed method, three sets of control experiments were designed to evaluate both objective and subjective aspects and verify the feasibility of the method presented in this paper. Compared with traditional assembly systems, the proposed method optimizes the recognition of human–robot collaborative assembly actions and applies them to intelligent control systems using digital-twin technology. This intelligent assembly method improves assembly efficiency and saves assembly time. It enables efficient and sustainable collaboration between humans and robots in assembly, leading to a positive and sustainable impact on the manufacturing industry.

Prediction of human initial operation situation in confined space with a multi-task deep neural network

Human-cyber-physical System for Operation in Nuclear Reactor Possessing Asymmetric Multi-Task Learning-Based Predicting Framework

Human risk recognition and prediction in manned submersible diving tasks driven by deep learning models

Learning-based 3D Occupancy Prediction for Autonomous Navigation in Occluded Environments

Predicting Standing Reach Postures using Deep Neural Networks

A Spatio-Temporal Transformer Network for Human Motion Prediction in Human-Robot Collaboration

Predicting the Posture of High-Rise Building Machines Based on Multivariate Time Series Neural Network Models

A fusion-based spiking neural network approach for predicting collaboration request in human-robot collaboration

Artificial Neural Network based Predictive Modeling of Operator Functional State

Deep learning based assembly process action recognition and progress prediction facing human-centric intelligent manufacturing

A novel spatio-temporal hybrid neural network for remaining useful life prediction

NLP-Based Approach for Predicting HMI State Sequences Towards Monitoring Operator Situational Awareness

Building Hybrid B-Spline And Neural Network Operators

A High-Precision Multi-Human Body Pose Estimation Approach for Near-Electricity Work in Substations

Smart industrial IoT empowered crowd sensing for safety monitoring in coal mine

Advancing human-robot collaboration: Predicting operator trajectories through AI and infrared imaging

Multitask Network for Joint Object Detection, Semantic Segmentation and Human Pose Estimation in Vehicle Occupancy Monitoring

Hybrid Convolutional Neural Network Approaches for Recognizing Collaborative Actions in Human–Robot Assembly Tasks

Network Generalization Prediction for Safety Critical Tasks in Novel Operating Domains

Hierarchical Deep Learning for Intention Estimation of Teleoperation Manipulation in Assembly Tasks

Geom-DeepONet: A Point-cloud-based Deep Operator Network for Field Predictions on 3D Parameterized Geometries