Abstract:Either RGB images or inertial signals have been used for the task of motion capture (mocap), but combining them together is a new and interesting topic. We believe that the combination is complementary and able to solve the inherent difficulties of using one modality input, including occlusions, extreme lighting/texture, and out-of-view for visual mocap and global drifts for inertial mocap. To this end, we propose a method that fuses monocular images and sparse IMUs for real-time human motion capture. Our method contains a dual coordinate strategy to fully explore the IMU signals with different goals in motion capture. To be specific, besides one branch transforming the IMU signals to the camera coordinate system to combine with the image information, there is another branch to learn from the IMU signals in the body root coordinate system to better estimate body poses. Furthermore, a hidden state feedback mechanism is proposed for both two branches to compensate for their own drawbacks in extreme input cases. Thus our method can easily switch between the two kinds of signals or combine them in different cases to achieve a robust mocap. %The two divided parts can help each other for better mocap results under different conditions. Quantitative and qualitative results demonstrate that by delicately designing the fusion method, our technique significantly outperforms the state-of-the-art vision, IMU, and combined methods on both global orientation and local pose estimation. Our codes are available for research at https://shaohua-pan.github.io/robustcap-page/.

Combined-type motion capturing system

A Combined Motion Capture System

Mixed motion capturing system and mixed motion capturing method

Mixed motion capture system and method

Human Motion Capture Using Wireless Inertial Sensors

Design of Human Motion Acquisition System Based on MIMU

Position and pose measurement technology of space combination ground experiment based on machine vision

Design and implementation of a wearable human motion capture system

Fusing Monocular Images and Sparse IMU Signals for Real-time Human Motion Capture

Design of Motion Capture System Based on MEMS.

Immersion type virtual reality system based on movement capture

Human-centric Application in Cyber-Physical System: an Inertial-based Motion Capture and Recognition System

Towards an Ubiquitous Motion Capture System Using Inertial MEMS Sensors and ZigBee Network

Real-Time Human Motion Capture Based on Wearable Inertial Sensor Networks

3D Deformation Capture Via a Configurable Self-Sensing IMU Sensor Network

Design and Implementation of Motion Capture System Based on Wireless Inertial Sensor

Portable motion capture device and use method thereof

Ubiquitous Human Body Motion Capture Using Micro-Sensors

A Human Motion Capture System Based on Inertial Sensing and a Complementary Filter

Design And Implementation Of The Imu Human Body Motion Tracking System

Movement capture data processing method, device and system