Abstract:In the context of sustainable manufacturing, efficient collaboration between humans and machines is crucial for improving assembly quality and efficiency. However, traditional methods for action recognition and human–robot collaborative assembly often face challenges such as low efficiency, low accuracy, and poor robustness. To solve such problems, this paper proposes an assembly action-recognition method based on a hybrid convolutional neural network. Firstly, an assembly action-recognition model is proposed using skeletal sequences and a hybrid convolutional neural network model combining Spatial Temporal Graph Convolutional Networks (ST-GCNs) and One-Dimensional Convolutional Neural Networks (1DCNNs) to sense and recognize human behavior actions during the assembly process. This model combines the joint spatial relationship and temporal information extraction ability of the ST-GCN model with the temporal feature extraction ability of the 1DCNN model. By incorporating Batch Normalization (BN) layers and Dropout layers, the generalization performance of the model is enhanced. Secondly, the model is validated on a self-constructed dataset of assembly actions, and the results show that the recognition accuracy of the model can reach 91.7%, demonstrating its superiority. Finally, a digital workshop application system based on digital twins is developed. To test the effectiveness of the proposed method, three sets of control experiments were designed to evaluate both objective and subjective aspects and verify the feasibility of the method presented in this paper. Compared with traditional assembly systems, the proposed method optimizes the recognition of human–robot collaborative assembly actions and applies them to intelligent control systems using digital-twin technology. This intelligent assembly method improves assembly efficiency and saves assembly time. It enables efficient and sustainable collaboration between humans and robots in assembly, leading to a positive and sustainable impact on the manufacturing industry.

Applying vision-guided graph neural networks for adaptive task planning in dynamic human robot collaborative scenarios

A Cascaded Deep Learning Framework for Real-time and Robust Grasp Planning

Robot-To-Human Handover with Obstacle Avoidance Via Continuous Time Recurrent Neural Network

A graph-based reinforcement learning-enabled approach for adaptive human-robot collaborative assembly operations

Integrating Uncertainty-Aware Human Motion Prediction into Graph-Based Manipulator Motion Planning

Robust Task Planning for Assembly Lines with Human-Robot Collaboration

KG-Planner: Knowledge-Informed Graph Neural Planning for Collaborative Manipulators

A stacked graph neural network with self-exciting process for robotic cognitive strategy reasoning in proactive human-robot collaborative assembly

Human-robot collaborative assembly task planning for mobile cobots based on deep reinforcement learning

Proactive Human-Robot Co-Assembly: Leveraging Human Intention Prediction and Robust Safe Control

A Vision-based Human Digital Twin Modelling Approach for Adaptive Human-Robot Collaboration

Hybrid Convolutional Neural Network Approaches for Recognizing Collaborative Actions in Human–Robot Assembly Tasks

Planning Coordinated Human-Robot Motions with Neural Network Full-Body Prediction Models

Towards Better Human Robot Collaboration with Robust Plan Recognition and Trajectory Prediction

Toward Proactive Human–Robot Collaborative Assembly: A Multimodal Transfer-Learning-Enabled Action Prediction Approach

Dynamic Inference on Graphs using Structured Transition Models

A vision-language-guided robotic action planning approach for ambiguity mitigation in human–robot collaborative manufacturing

To Lead or to Follow? Adaptive Robot Task Planning in Human-Robot Collaboration

Neural-Guided RuntimePrediction of Planners for Improved Motion and Task Planning with Graph Neural Networks

Coordinate Invariant User-Guided Constrained Path Planning with Reactive Rapidly Expanding Plane-Oriented Escaping Trees

Graph Wasserstein Autoencoder-Based Asymptotically Optimal Motion Planning with Kinematic Constraints for Robotic Manipulation.