Abstract:Background Model-based 3D pose estimation has been widely used in many 3D human motion analysis applications, in which vision-based and inertial-based are two distinct lines. Multi-view images in a vision-based markerless capture system provide essential data for motion analysis, but erroneous estimates still occur due to ambiguities, occlusion, or noise in images. Besides, the multi-view setting is hard for the application in the wild. Although inertial measurement units (IMUs) can obtain accurate direction without occlusion, they are usually susceptible to magnetic field interference and drifts. Hybrid motion capture has drawn the attention of researchers in recent years. Existing 3D pose estimation methods jointly optimize the parameters of the 3D pose by minimizing the discrepancy between the image and IMU data. However, these hybrid methods still suffer from the issues such as complex peripheral devices, sensitivity to initialization, and slow convergence. Methods This article presents an approach to improve 3D human pose estimation by fusing a single image with sparse inertial measurement units (IMUs). Based on a dual-stream feature extract network, we design a model-attention network with a residual module to closely couple the dual-modal feature from a static image and sparse inertial measurement units. The final 3D pose and shape parameters are directly obtained by a regression strategy. Results Extensive experiments are conducted on two benchmark datasets for 3D human pose estimation. Compared to state-of-the-art methods, the per vertex error (PVE) of human mesh reduces by 9.4 mm on Total Capture dataset and the mean per joint position error (MPJPE) reduces by 7.8 mm on the Human3.6M dataset. The quantitative comparison demonstrates that the proposed method could effectively fuse sparse IMU data and images and improve pose accuracy.

Image-based three-dimensional human pose recovery by multiview locality-sensitive sparse retrieval

3D human pose recovery from image by efficient visual feature selection

Multimodal Deep Autoencoder for Human Pose Recovery

Marker-Less 3d Human Motion Capture With Monocular Image Sequence And Height-Maps

Reconstructing 3D human pose and shape from a single image and sparse IMUs

Multitask Autoencoder Model for Recovering Human Poses.

Motion Capture Research: 3D Human Pose Recovery Based on RGB Video Sequences

Implicit 3D Human Mesh Recovery using Consistency with Pose and Shape from Unseen-view

A novel joint points and silhouette-based method to estimate 3D human pose and shape

3D Human pose estimation from video via multi-scale multi-level spatial temporal features

3D Human Pose and Shape Estimation with Dense Correspondence from a Single Depth Image

View Consistency Aware Holistic Triangulation for 3D Human Pose Estimation

Exploring Latent Cross-Channel Embedding for Accurate 3D Human Pose Reconstruction in a Diffusion Framework

High-throughput investigation of osteoblast response to polymer crystallinity: influence of nanometer-scale roughness on proliferation.

Robust Estimation of 3D Human Poses from a Single Image

A Survey on Monocular 3D Human Pose Estimation

Sparseness Meets Deepness: 3D Human Pose Estimation from Monocular Video

Weakly-Supervised 3D Human Pose Learning via Multi-view Images in the Wild

Lifting by Image -- Leveraging Image Cues for Accurate 3D Human Pose Estimation

3D Pose Estimation Based on Reinforce Learning for 2D Image-Based 3D Model Retrieval

Efficient 3D Recovery of Human Motion in Monocular Video