Abstract:Although commercial motion-capture systems have been widely used in various applications, the complex setup limits their application scenarios for ordinary consumers. To overcome the drawbacks of wearability, human posture reconstruction based on a few wearable sensors have been actively studied in recent years. In this paper, we propose a deep-learning-based sparse inertial sensor human posture reconstruction method. This method uses bidirectional recurrent neural network (Bi-RNN) to build an a priori model from a large motion dataset to build human motion, thereby the low-dimensional motion measurements are mapped to whole-body posture. To improve the motion reconstruction performance for specific application scenarios, two fundamental problems in the model construction are investigated: training data selection and sparse sensor placement. The problem of deep-learning training data selection is to select independent and identically distributed (IID) data for a certain scenario from the accumulated imbalanced motion dataset with sufficient information. We formulate the data selection into an optimization problem to obtain continuous and IID data segments, which comply with a small reference dataset collected from the target scenario. A two-step heuristic algorithm is proposed to solve the data selection problem. On the other hand, the optimal sensor placement problem is studied to exploit most information from partial observation of human movement. A method for evaluating the motion information amount of any group of wearable inertial sensors based on mutual information is proposed, and a greedy searching method is adopted to obtain the approximate optimal sensor placement of a given sensor number, so that the maximum motion information and minimum redundancy is achieved. Finally, the human posture reconstruction performance is evaluated with different training data and sensor placement selection methods, and experimental results show that the proposed method takes advantages in both posture reconstruction accuracy and model training time. In the 6 sensors configuration, the posture reconstruction errors of our model for walking, running, and playing basketball are 7.25°, 8.84°, and 14.13°, respectively.

Motion Reconstruction from Sparse Accelerometer Data Using PLSR

Mpose: Environment- and Subject-Agnostic 3D Skeleton Posture Reconstruction Leveraging a Single Mmwave Device

SparsePoser: Real-time Full-body Motion Reconstruction from Sparse Data

Training Data Selection and Optimal Sensor Placement for Deep-Learning-Based Sparse Inertial Sensor Human Posture Reconstruction

Leveraging depth cameras and wearable pressure sensors for full-body kinematics and dynamics capture

3D Upper Limb Motion Modeling and Estimation Using Wearable Micro-sensors

Full-body Human Motion Reconstruction with Sparse Joint Tracking Using Flexible Sensors

Real-Time Human Motion Capture Based on Wearable Inertial Sensor Networks

Predicting Missing Markers In Human Motion Capture Using L1-Sparse Representation

Real-time Physics-based Motion Capture with Sparse Sensors

Markerless Motion Capture Using Appearance and Inertial Data

A Scalable and Wearable Self-Sensing IMU Sensor Network for Personalized Human Motion and Deformation Capture

Wearable Sensors for 3D Upper Limb Motion Modeling and Ubiquitous Estimation

EM-POSE: 3D Human Pose Estimation from Sparse Electromagnetic Trackers

Physical Inertial Poser (PIP): Physics-aware Real-time Human Motion Tracking from Sparse Inertial Sensors

Ultra Inertial Poser: Scalable Motion Capture and Tracking from Sparse Inertial Sensors and Ultra-Wideband Ranging

Full-Body Motion Reconstruction with Sparse Sensing from Graph Perspective

DiffusionPoser: Real-time Human Motion Reconstruction From Arbitrary Sparse Sensors Using Autoregressive Diffusion

PE-DLS: a novel method for performing real-time full-body motion reconstruction in VR based on Vive trackers

Synergetic Reconstruction from 2D Pose and 3D Motion for Wide-Space Multi-Person Video Motion Capture in the Wild

A wearable motion capture device able to detect dynamic motion of human limbs