Abstract:Recently, with the growing popularity of Convolutional Neural Networks (CNN), major progress has been made in human pose estimation. However, due to the lack of consideration of human body structure, the current methods still suffer from occlusion, complex background, and large pose variations. Human parsing, as a highly relevant task, can provide useful semantic information of body parts for pose estimation. In this paper, we propose a novel convolutional network, which consists of a pose encoder and a parsing encoder in parallel, to combine parsing information with body structure information for effectively assisting human pose estimation in unconstrained environment. The pose encoder can extract discriminative features that contain rich structure information by incorporating a multi-branch parallel module and a hierarchical connection architecture into hourglass network. On the other hand, the parsing encoder can obtain valuable parsing information that serves as extra constraints to help improve the performance of pose estimation. Then, the generated pose features and parsing features will be integrated together for heatmap prediction. In addition, we apply a joint classification loss for preserving the structural consistency between keypoints and body parts. By correctly localizing the joints in their corresponding body parts, the network can better optimize the spatial relationship between different keypoints, thus reducing the errors of estimation. It is worth mentioning that the proposed modules can be introduced into most of the existing methods. The experimental results on the extended PASCAL-Person-Part and LSP datasets demonstrate the effectiveness of our model, and the performance can be greatly improved by utilizing parsing information.

Learning hierarchical poselets for human parsing

Discriminative Hierarchical Part-Based Models for Human Parsing and Action Recognition.

Pose-Guided Hierarchical Semantic Decomposition and Composition for Human Parsing

Pose-Guided Human Parsing with Deep Learned Features

Towards Unified Human Parsing and Pose Estimation

A Deformable Mixture Parsing Model with Parselets

Hierarchical Human Parsing with Typed Part-Relation Reasoning

Pose-Guided Human Parsing By An And/Or Graph Using Pose-Context Features

Weakly and Semi Supervised Human Body Part Parsing Via Pose-Guided Knowledge Transfer

Hierarchical Human Semantic Parsing With Comprehensive Part-Relation Modeling

Human Pose Estimation Via Parse Graph of Body Structure

Kinematic Skeleton Graph Augmented Network for Human Parsing

Enhanced Human Parsing with Multiple Feature Fusion and Augmented Pose Model

Differentiable Multi-Granularity Human Parsing

Self-supervised Structure-Sensitive Learning for Human Parsing

Deep Hierarchical Human Semantic Parsing

Differentiable Multi-Granularity Human Representation Learning for Instance-Aware Human Semantic Parsing

Combining Parsing Information with Joint Structure for Human Pose Estimation.

Learning Visual Symbols for Parsing Human Poses in Images

Human Parsing Using Stochastic And-or Grammars and Rich Appearances

Finer-Net: Cascaded Human Parsing with Hierarchical Granularity