Konstantinos Papoutsakis,Nikolaos Bakalos,Konstantinos Fragkoulis,Athena Zacharia,Georgia Kapetadimitri,Maria Pateraki
Abstract:This paper introduces a vision-based framework for capturing and understanding human behavior in industrial assembly lines, focusing on car door manufacturing. The framework leverages advanced computer vision techniques to estimate workers' locations and 3D poses and analyze work postures, actions, and task progress. A key contribution is the introduction of the CarDA dataset, which contains domain-relevant assembly actions captured in a realistic setting to support the analysis of the framework for human pose and action analysis. The dataset comprises time-synchronized multi-camera RGB-D videos, motion capture data recorded in a real car manufacturing environment, and annotations for EAWS-based ergonomic risk scores and assembly activities. Experimental results demonstrate the effectiveness of the proposed approach in classifying worker postures and robust performance in monitoring assembly task progress.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to understand and analyze human behaviors in industrial assembly lines, especially the behaviors of workers in the car door manufacturing process. Specifically, the paper aims to capture and understand the positions, 3D postures, working postures, movements, and task progress of workers through visual technology, thereby improving productivity and safety. The following are the key problems that the paper attempts to solve:
1. **Real - time monitoring of worker behaviors**: In a complex industrial environment, how to monitor and evaluate the physical postures and operational behaviors of workers in real - time to identify potential health risks and inefficient links in the work process.
2. **Severe long - term human occlusion problem**: In the actual assembly line, due to the cooperation of multiple people and the presence of other objects, the body parts of workers may be occluded for a long time. How to accurately estimate the human posture in this situation is a challenge.
3. **Understanding of complex human assembly actions**: The actions performed by workers on the assembly line are often relatively complex. How to understand these actions and classify them is also a difficult problem.
4. **Lack of data sets**: Most of the currently available data sets focus on activities of daily living, and there are few data sets specifically for manufacturing actions, especially data sets that record large - scale body movements from an exocentric perspective and the interaction between people and car doors.
To solve these problems, the author proposes a vision - based framework and introduces the CarDA data set. This data set contains time - synchronized multi - camera RGB - D videos, motion capture data, and pose grid annotations based on EAWS (European Assessment Worksheet) recorded in a real - car - manufacturing environment. Through these data, researchers can train and evaluate models for human pose estimation, posture assessment, and action monitoring to ensure their applicability and effectiveness in the actual industrial environment.
### Main contributions of the paper
1. **Introduction of the CarDA data set**: A comprehensive multi - modal data set that contains time - synchronized RGB - D videos and motion capture data, records the real car door assembly process, and provides annotations of 3D human postures, posture risk scores, and assembly activities.
2. **Development of a vision - based framework**: This framework utilizes advanced computer vision techniques to estimate the positions and 3D postures of workers, analyze working postures and actions, and monitor the progress of assembly tasks.
3. **Proposing effective deep - learning methods**: Using graph - neural - network - based methods to identify worker postures in an industrial environment, especially performing well in dealing with occlusion problems.
Through these contributions, the paper provides a powerful tool for understanding human behaviors in industrial assembly lines, which helps to improve production efficiency and worker safety.