Abstract:Despite the significant improvement in the performance of monocular pose estimation approaches and their ability to generalize to unseen environments, multi-view approaches are often lagging behind in terms of accuracy and are specific to certain datasets. This is mainly due to the fact that (1) contrary to real-world single-view datasets, multi-view datasets are often captured in controlled environments to collect precise 3D annotations, which do not cover all real-world challenges, and (2) the model parameters are learned for specific camera setups. To alleviate these problems, we propose a two-stage approach to detect and estimate 3D human poses, which separates single-view pose detection from multi-view 3D pose estimation. This separation enables us to utilize each dataset for the right task, i.e. single-view datasets for constructing robust pose detection models and multi-view datasets for constructing precise multi-view 3D regression models. In addition, our 3D regression approach only requires 3D pose data and its projections to the views for building the model, hence removing the need for collecting annotated data from the test setup. Our approach can therefore be easily generalized to a new environment by simply projecting 3D poses into 2D during training according to the camera setup used at test time. As 2D poses are collected at test time using a single-view pose detector, which might generate inaccurate detections, we model its characteristics and incorporate this information during training. We demonstrate that incorporating the detector's characteristics is important to build a robust 3D regression model and that the resulting regression model generalizes well to new multi-view environments. Our evaluation results show that our approach achieves competitive results on the Human3.6M dataset and significantly improves results on a multi-view clinical dataset that is the first multi-view dataset generated from live surgery recordings.

Harvesting Multiple Views for Marker-less 3 D Human Pose Annotations Supplementary Material

Harvesting Multiple Views for Marker-less 3D Human Pose Annotations.

Marker-Less 3d Human Motion Capture With Monocular Image Sequence And Height-Maps

Markerless Multi-view 3D Human Pose Estimation: a survey

Part-Aware Measurement for Robust Multi-View Multi-Human 3D Pose Estimation and Tracking

Two Views Are Better than One: Monocular 3D Pose Estimation with Multiview Consistency

View Consistency Aware Holistic Triangulation for 3D Human Pose Estimation

Exemplar Fine-Tuning for 3D Human Model Fitting Towards In-the-Wild 3D Human Pose Estimation

Fast and Robust Multi-Person 3D Pose Estimation from Multiple Views

Direct Multi-view Multi-person 3D Pose Estimation

Self-learning Canonical Space for Multi-view 3D Human Pose Estimation

New multi-view human motion capture framework

Towards Accurate Markerless Human Shape and Pose Estimation over Time

Unsupervised View-Invariant Human Posture Representation

A generalizable approach for multi-view 3D human pose regression

Toward Marker-free 3D Pose Estimation in Lifting: A Deep Multi-view Solution

3D Human Pose Estimation Based on Wearable IMUs and Multiple Camera Views

Human-M3: A Multi-view Multi-modal Dataset for 3D Human Pose Estimation in Outdoor Scenes

3D Human pose estimation from video via multi-scale multi-level spatial temporal features

Multiple View Geometry Transformers for 3D Human Pose Estimation

SoloPose: One-Shot Kinematic 3D Human Pose Estimation with Video Data Augmentation