Abstract:Either RGB images or inertial signals have been used for the task of motion capture (mocap), but combining them together is a new and interesting topic. We believe that the combination is complementary and able to solve the inherent difficulties of using one modality input, including occlusions, extreme lighting/texture, and out-of-view for visual mocap and global drifts for inertial mocap. To this end, we propose a method that fuses monocular images and sparse IMUs for real-time human motion capture. Our method contains a dual coordinate strategy to fully explore the IMU signals with different goals in motion capture. To be specific, besides one branch transforming the IMU signals to the camera coordinate system to combine with the image information, there is another branch to learn from the IMU signals in the body root coordinate system to better estimate body poses. Furthermore, a hidden state feedback mechanism is proposed for both two branches to compensate for their own drawbacks in extreme input cases. Thus our method can easily switch between the two kinds of signals or combine them in different cases to achieve a robust mocap. %The two divided parts can help each other for better mocap results under different conditions. Quantitative and qualitative results demonstrate that by delicately designing the fusion method, our technique significantly outperforms the state-of-the-art vision, IMU, and combined methods on both global orientation and local pose estimation. Our codes are available for research at https://shaohua-pan.github.io/robustcap-page/.

Mixed motion capturing system and mixed motion capturing method

Mixed motion capture system and method

A Combined Motion Capture System

Combined-type motion capturing system

Human Motion Capture Using Wireless Inertial Sensors

A Low-Cost Drift-Free Optical-Inertial Hybrid Motion Capture System for High-Precision Human Pose Detection

A Lightweight and Scalable Visual-Inertial Motion Capture System Using Fiducial Markers

Approach of Scattered Data Processing for Optical Motion Capture

Markerless Motion Capture Using Appearance and Inertial Data

Real-Time Human Motion Capture Based on Wearable Inertial Sensor Networks

Automatic Marker Registration and Prediction Algorithm for Optical Motion Capture System

Optical Unmarked Motion Capture Technology Based on Depth Network and Binocular Vision

Towards Human Motion Tracking: Multi-Sensory IMU/TOA Fusion Method and Fundamental Limits

An Effective Method of Capture Data Processing for Optical Motion

Portable motion capture device and use method thereof

Fusing Monocular Images and Sparse IMU Signals for Real-time Human Motion Capture

Design of Motion Capture System Based on MEMS.

Ambulatory Real-Time Micro-Sensor Motion Capture

Multi mobile robot movement of human features Multisensor Tracking Method

Design and Implementation of Motion Capture System Based on Wireless Inertial Sensor

Generic Content-Based Retrieval of Marker-Based Motion Capture Data.