Abstract:Landmark/pose estimation in single monocular images has received much effort in computer vision due to its important applications. It remains a challenging task when input images come with severe occlusions caused by, e.g., adverse camera views. Under such circumstances, biologically implausible pose predictions may be produced. In contrast, human vision is able to predict poses by exploiting geometric constraints of landmark point inter-connectivity. To address the problem, by incorporating priors about the structure of pose components, we propose a novel structure-aware fully convolutional network to implicitly take such priors into account during training of the deep network. Explicit learning of such constraints is typically challenging. Instead, inspired by how human identifies implausible poses, we design discriminators to distinguish the real poses from the fake ones (such as biologically implausible ones). If the pose generator G generates results that the discriminator fails to distinguish from real ones, the network successfully learns the priors. Training of the network follows the strategy of conditional Generative Adversarial Networks (GANs). The effectiveness of the proposed network is evaluated on three pose-related tasks: 2D human pose estimation, 2D facial landmark estimation and 3D human pose estimation. The proposed approach significantly outperforms several state-of-the-art methods and almost always generates plausible pose predictions, demonstrating the usefulness of implicit learning of structures using GANs.

Structure-aware Person Image Generation with Pose Decomposition and Semantic Correlation.

GLocal: Global Graph Reasoning and Local Structure Transfer for Person Image Generation

Hierarchical Generation Of Human Pose With Part-Based Layer Representation

Deep Image Spatial Transformation for Person Image Generation

Unsupervised Person Image Generation with Semantic Parsing Transformation

Human Pose Estimation Using Deep Structure Guided Learning.

SCRN: Stepwise Change and Refine Network Based Semantic Distribution for Human Pose Transfer

Learning Semantic Person Image Generation by Region-Adaptive Normalization

Deep Spatial Transformation for Pose-Guided Person Image Generation and Animation.

Pose Guided Image Generation from Misaligned Sources Via Residual Flow Based Correction

Pose with style

Correspondence Learning for Controllable Person Image Generation

Exploiting appearance transfer and multi-scale context for efficient person image generation

Pose Guided Image Generation from Misaligned Sources via Residual Flow Based Correction

Pose- and Attribute-consistent Person Image Synthesis

Flow-guided attention deformation for person image generation

Unpaired semantic neural person image synthesis

Verbal-Person Nets: Pose-Guided Multi-Granularity Language-to-Person Generation

Pose Generator ( G ) : Head : R arm : L arm : Chest : R leg : L leg Plausible Pose

Where and Who? Automatic Semantic-Aware Person Composition

Multi-Scale Structure-Aware Network for Human Pose Estimation