Abstract:Video-based human pose estimation (VHPE) is a vital yet challenging task. While deep learning methods have made significant progress for the VHPE, most approaches to this task implicitly model the long-range interaction between joints by enlarging the receptive field of the convolution. Unlike prior methods, we design a lightweight and plug-and-play joint relation extractor (JRE) to model the associative relationship between joints explicitly and automatically. The JRE takes the pseudo heatmaps of joints as input and calculates the similarity between pseudo heatmaps. In this way, the JRE flexibly learns the relationship between any two joints, allowing it to learn the rich spatial configuration of human poses. Moreover, the JRE can infer invisible joints according to the relationship between joints, which is beneficial for the model to locate occluded joints. Then, combined with temporal semantic continuity modeling, we propose a Relation-based Pose Semantics Transfer Network (RPSTN) for video-based human pose estimation. Specifically, to capture the temporal dynamics of poses, the pose semantic information of the current frame is transferred to the next with a joint relation guided pose semantics propagator (JRPSP). The proposed model can transfer the pose semantic features from the non-occluded frame to the occluded frame, making our method robust to the occlusion. Furthermore, the proposed JRE module is also suitable for image-based human pose estimation. The proposed RPSTN achieves state-of-the-art results on the video-based Penn Action dataset, Sub-JHMDB dataset, and PoseTrack2018 dataset. Moreover, the proposed JRE improves the performance of backbones on the image-based COCO2017 dataset. Code is available at <a class="link-external link-https" href="https://github.com/YHDang/pose-estimation" rel="external noopener nofollow">this https URL</a>.

Joint Relation Based Human Pose Estimation

Human Pose Estimation Based on Human Limbs.

A Limb-Based Graphical Model for Human Pose Estimation.

Relation-Based Associative Joint Location for Human Pose Estimation in Videos

Localization of Human 3D Joints Based on Binocular Vision

3D Human Pose Estimation in Motion Based on Multi-Stage Regression

Learning Joint Structure for Human Pose Estimation

Position Constrained Network for 3D Human Pose Estimation

Human Joints Estimation System of Rescue Robot with Occlusion

A Joint Relationship Aware Neural Network for Single-Image 3D Human Pose Estimation

Occluded Human Pose Estimation based on Limb Joint Augmentation

3D Joints Estimation of the Human Body in Single-Frame Point Cloud

Robust Estimation of 3D Human Poses from a Single Image

Joint-Limb Compound Triangulation with Co-Fixing for Stereoscopic Human Pose Estimation

3D Human Pose Estimation Based on Multi View Information Fusion

Estimation of 3D Human Pose Using Prior Knowledge

Human Pose Estimation using Global and Local Normalization

3D joints estimation of human body using part segmentation

A novel joint points and silhouette-based method to estimate 3D human pose and shape

Human Joint Angle Estimation Using Deep Learning-Based Three-Dimensional Human Pose Estimation for Application in a Real Environment

Towards Locality Similarity Preserving to 3D Human Pose Estimation.