Abstract:Background Model-based 3D pose estimation has been widely used in many 3D human motion analysis applications, in which vision-based and inertial-based are two distinct lines. Multi-view images in a vision-based markerless capture system provide essential data for motion analysis, but erroneous estimates still occur due to ambiguities, occlusion, or noise in images. Besides, the multi-view setting is hard for the application in the wild. Although inertial measurement units (IMUs) can obtain accurate direction without occlusion, they are usually susceptible to magnetic field interference and drifts. Hybrid motion capture has drawn the attention of researchers in recent years. Existing 3D pose estimation methods jointly optimize the parameters of the 3D pose by minimizing the discrepancy between the image and IMU data. However, these hybrid methods still suffer from the issues such as complex peripheral devices, sensitivity to initialization, and slow convergence. Methods This article presents an approach to improve 3D human pose estimation by fusing a single image with sparse inertial measurement units (IMUs). Based on a dual-stream feature extract network, we design a model-attention network with a residual module to closely couple the dual-modal feature from a static image and sparse inertial measurement units. The final 3D pose and shape parameters are directly obtained by a regression strategy. Results Extensive experiments are conducted on two benchmark datasets for 3D human pose estimation. Compared to state-of-the-art methods, the per vertex error (PVE) of human mesh reduces by 9.4 mm on Total Capture dataset and the mean per joint position error (MPJPE) reduces by 7.8 mm on the Human3.6M dataset. The quantitative comparison demonstrates that the proposed method could effectively fuse sparse IMU data and images and improve pose accuracy.

Markerless Human Pose Estimation Using Image Features and Extremal Contour

Marker-Less 3d Human Motion Capture With Monocular Image Sequence And Height-Maps

Markerless Human Body Motion Capture Using Multiple Cameras

Model-Based Markerless Human Body Motion Capture using Multiple Cameras

Model-based markerless human body motion capture using active contour

Markerless motion capture based on diffusion geometry and silhouettes

Towards Accurate Markerless Human Shape and Pose Estimation over Time

A new hierarchical method for markerless human pose estimation

Optimization and Soft Constraints for Human Shape and Pose Estimation Based on a 3D Morphable Model

3D Human Body Shape and Pose Estimation from Depth Image.

A New Algorithm for Human Motion Capture via 3D Active Contours

Reconstructing 3D human pose and shape from a single image and sparse IMUs

Markerless Human Body Motion Capture Using Markov Random Field and Dynamic Graph Cuts

MODEL-BASEDMARKERLESS HUMAN BODY MOTION CAPTURE USINGMULTIPLE CAMERAS

Using compact distance energy model to recover human pose in markerless motion capture

Marker-less Motion Capture Technology Based on Binocular Stereo Vision and Deep Learning

Human model adaptation for multiview markerless motion capture

3D Human Pose Estimation Based on Multi View Information Fusion

3D Human Pose Estimation in Motion Based on Multi-Stage Regression

Human Body Motion Capture Via 3d Graph-Cuts

Voxel Data Based Marker-less Human Motion Capture Using Geometry Model