Abstract:International Journal of Distributed Sensor Networks, Volume 18, Issue 5, May 2022. Skeleton tracking based on multiple Kinects data fusion has been proved to have better accuracy and robustness than single Kinect. However, previous works did not consider the inconsistency of tracking accuracy in the tracking field of Kinect and the self-occlusion of human body in assembly operation, which are of vital importance to the fusion performance of the multiple Kinects data in assembly task simulation. In this work, we developed a multi-Kinect fusion algorithm to achieve robust full-body tracking in virtual reality (VR)-aided assembly simulation. Two reliability functions are first applied to evaluate the tracking confidences reflecting the impacts of the position-related error and the self-occlusion error on the tracked skeletons. Then, the tracking skeletons from multiple Kinects are fused based on weighted arithmetic average and generalized covariance intersection. To evaluate the tracking confidence, the ellipsoidal surface fitting was used to model the tracking accuracy distribution of Kinect, and the relations between the user-Kinect crossing angles and the influences of the self-occlusion on the tracking of different parts of body were studied. On the basis, the two reliability functions were developed. We implemented a prototype system leveraging six Kinects and applied the distributed computing in the system to improve the computing efficiency. Experiment results showed that the proposed algorithm has superior fusion performance compared to the peer works.

A Novel Vision-Based Human Motion Capture System Using Dual-Kinect

Development of a Self-Calibrated Motion Capture System by Nonlinear Trilateration of Multiple Kinects v2

Inertial-Kinect Fusion for Robot Navigation based on the Extended Kalman Filter

Data Fusion of Human Skeleton Joint Tracking Using Two Kinect Sensors and Extended Set Membership Filter

Human Motion Tracking by Multiple RGBD Cameras.

Human Motion Capture Using Wireless Inertial Sensors

A new approach for tracking human body movements by kinect sensor

Real-Time Human Motion Capture Based on Wearable Inertial Sensor Networks

Leveraging Two Kinect Sensors for Accurate Full-Body Motion Capture

Tracking Human-like Natural Motion Using Deep Recurrent Neural Networks

Optical Unmarked Motion Capture Technology Based on Depth Network and Binocular Vision

Performance Capture Of Interacting Characters With Handheld Kinect

Fast IMU-based Dual Estimation of Human Motion and Kinematic Parameters via Progressive In-Network Computing

Real-time 3D Human Motion Capture from Binocular Stereo Vision and Its Use on Human Body Animation

Multi-Kinects fusion for full-body tracking in virtual reality-aided assembly simulation

Multi-person Vision-Based Head Detector for Markerless Human Motion Capture

Data fusion of multiple kinect sensors for a rehabilitation system.

Human Performance Capture Using Multiple Handheld Kinects

Human gait capture by tracking joint motion based on Kalman Filter

Marker-less Motion Capture Technology Based on Binocular Stereo Vision and Deep Learning

A Markless 3D Human Motion Data Acquisition Method Based on the Binocular Stereo Vision and Lightweight Open Pose Algorithm