Abstract:Background Model-based 3D pose estimation has been widely used in many 3D human motion analysis applications, in which vision-based and inertial-based are two distinct lines. Multi-view images in a vision-based markerless capture system provide essential data for motion analysis, but erroneous estimates still occur due to ambiguities, occlusion, or noise in images. Besides, the multi-view setting is hard for the application in the wild. Although inertial measurement units (IMUs) can obtain accurate direction without occlusion, they are usually susceptible to magnetic field interference and drifts. Hybrid motion capture has drawn the attention of researchers in recent years. Existing 3D pose estimation methods jointly optimize the parameters of the 3D pose by minimizing the discrepancy between the image and IMU data. However, these hybrid methods still suffer from the issues such as complex peripheral devices, sensitivity to initialization, and slow convergence. Methods This article presents an approach to improve 3D human pose estimation by fusing a single image with sparse inertial measurement units (IMUs). Based on a dual-stream feature extract network, we design a model-attention network with a residual module to closely couple the dual-modal feature from a static image and sparse inertial measurement units. The final 3D pose and shape parameters are directly obtained by a regression strategy. Results Extensive experiments are conducted on two benchmark datasets for 3D human pose estimation. Compared to state-of-the-art methods, the per vertex error (PVE) of human mesh reduces by 9.4 mm on Total Capture dataset and the mean per joint position error (MPJPE) reduces by 7.8 mm on the Human3.6M dataset. The quantitative comparison demonstrates that the proposed method could effectively fuse sparse IMU data and images and improve pose accuracy.

3DPMesh: An enhanced and novel approach for the reconstruction of 3D human meshes from a single 2D image

3D Real Human Reconstruction Via Multiple Low-Cost Depth Cameras.

3D Human Reconstruction from A Single Depth Image

MeshPose: Unifying DensePose and 3D Body Mesh reconstruction

MI-Mesh: 3D Human Mesh Construction by Fusing Image and Millimeter Wave

Personalized 3D Human Pose and Shape Refinement

MI-Mesh

Co-Evolution of Pose and Mesh for 3D Human Body Estimation from Video

3D Human Mesh Estimation from Virtual Markers

Skeleton2Mesh - Kinematics Prior Injected Unsupervised Human Mesh Recovery.

Human Mesh Recovery from Monocular Images via a Skeleton-disentangled Representation

Sequential 3D Human Pose and Shape Estimation from Point Clouds

HyperGraph Based Human Mesh Hierarchical Representation and Reconstruction from a Single Image.

Reconstructing 3D human pose and shape from a single image and sparse IMUs

Reconstructing 3D Human Pose from RGB-D Data with Occlusions

Detailed Human Shape Estimation From A Single Image By Hierarchical Mesh Deformation

Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Pose

3D Human Pose and Shape Estimation with Dense Correspondence from a Single Depth Image

Moulding Humans: Non-parametric 3D Human Shape Estimation from Single Images

Detailed 3D Human Body Reconstruction from a Single Image Based on Mesh Deformation

FAMOUS: High-Fidelity Monocular 3D Human Digitization Using View Synthesis