Abstract:Although commercial motion-capture systems have been widely used in various applications, the complex setup limits their application scenarios for ordinary consumers. To overcome the drawbacks of wearability, human posture reconstruction based on a few wearable sensors have been actively studied in recent years. In this paper, we propose a deep-learning-based sparse inertial sensor human posture reconstruction method. This method uses bidirectional recurrent neural network (Bi-RNN) to build an a priori model from a large motion dataset to build human motion, thereby the low-dimensional motion measurements are mapped to whole-body posture. To improve the motion reconstruction performance for specific application scenarios, two fundamental problems in the model construction are investigated: training data selection and sparse sensor placement. The problem of deep-learning training data selection is to select independent and identically distributed (IID) data for a certain scenario from the accumulated imbalanced motion dataset with sufficient information. We formulate the data selection into an optimization problem to obtain continuous and IID data segments, which comply with a small reference dataset collected from the target scenario. A two-step heuristic algorithm is proposed to solve the data selection problem. On the other hand, the optimal sensor placement problem is studied to exploit most information from partial observation of human movement. A method for evaluating the motion information amount of any group of wearable inertial sensors based on mutual information is proposed, and a greedy searching method is adopted to obtain the approximate optimal sensor placement of a given sensor number, so that the maximum motion information and minimum redundancy is achieved. Finally, the human posture reconstruction performance is evaluated with different training data and sensor placement selection methods, and experimental results show that the proposed method takes advantages in both posture reconstruction accuracy and model training time. In the 6 sensors configuration, the posture reconstruction errors of our model for walking, running, and playing basketball are 7.25°, 8.84°, and 14.13°, respectively.

Mpose: Environment- and Subject-Agnostic 3D Skeleton Posture Reconstruction Leveraging a Single Mmwave Device

Mmskeleton: 3D Human Skeleton Estimation Using Millimeter Wave Radar Sparse Point Clouds

Mmhand: 3D Hand Pose Estimation Leveraging Mmwave Signals

MDPose: Human Skeletal Motion Reconstruction Using WiFi Micro-Doppler Signatures

Towards 3D Human Pose Construction Using Wifi.

mm-Pose: Real-Time Human Skeletal Posture Estimation Using mmWave Radars and CNNs

RF-MvCare: RFID-based 3D Human Pose Estimation for Non-lineof-sight Conditions

RF-based 3D skeletons

EM-POSE: 3D Human Pose Estimation from Sparse Electromagnetic Trackers

mmPose-FK: A Forward Kinematics Approach to Dynamic Skeletal Pose Estimation Using mmWave Radars

Dynamic Human Body Reconstruction and Motion Tracking with Low-Cost Depth Cameras

3D Upper Body Reconstruction with Sparse Soft Sensors

3D Human Pose Estimation for Free-from and Moving Activities Using WiFi

3 D Adaptive Reconstruction of Human Motion From Multi-Sensors

MI-Mesh: 3D Human Mesh Construction by Fusing Image and Millimeter Wave

mmPose-NLP: A Natural Language Processing Approach to Precise Skeletal Pose Estimation Using mmWave Radars

From Point to Space: 3D Moving Human Pose Estimation Using Commodity WiFi

M 3 Track

Reconstructing Human Pose from Inertial Measurements: A Generative Model-based Compressive Sensing Approach

Accurate Human Pose Estimation using RF Signals

Training Data Selection and Optimal Sensor Placement for Deep-Learning-Based Sparse Inertial Sensor Human Posture Reconstruction