Abstract:Despite the noticeable progress in perceptual tasks like detection, instance segmentation and human parsing, computers still perform unsatisfactorily on visually understanding humans in crowded scenes, such as group behavior analysis, person re-identification, e-commerce, media editing, video surveillance, autonomous driving and virtual reality, etc. To perform well, models need to comprehensively perceive the semantic information and the differences between instances in a multi-human image, which is recently defined as the multi-human parsing task. In this paper, we first present a new large-scale database “Multi-human Parsing (MHP v2.0)” for algorithm development and evaluation to advance the research on understanding humans in crowded scenes. MHP v2.0 contains 25,403 elaborately annotated images with 58 fine-grained semantic category labels and 16 dense pose key point labels, involving 2–26 persons per image captured in real-world scenes from various viewpoints, poses, occlusion, interactions and background. We further propose a novel deep Nested Adversarial Network (NAN) model for multi-human parsing. NAN consists of three Generative Adversarial Network-like sub-nets, respectively performing semantic saliency prediction, instance-agnostic parsing and instance-aware clustering. These sub-nets form a nested structure and are carefully designed to learn jointly in an end-to-end way. NAN consistently outperforms existing state-of-the-art solutions on our MHP and several other datasets, including MHP v1.0, PASCAL-Person-Part and Buffy. NAN serves as a strong baseline to shed light on generic instance-level semantic part prediction and drive the future research on multi-human parsing. With the above innovations and contributions, we have organized the CVPR 2018 Workshop on Visual Understanding of Humans in Crowd Scene (VUHCS 2018) and the Fine-Grained Multi-human Parsing and Pose Estimation Challenge. These contributions together significantly benefit the community. Code and pre-trained models are available at https://github.com/ZhaoJ9014/Multi-Human-Parsing_MHP.

Look Into Multi-Person: A New Benchmark For Pose Estimation And Human Parsing

Look into Person: Joint Body Parsing & Pose Estimation Network and A New Benchmark

MOtion Human Parsing - A New Benchmark for 3D Human Parsing.

Exploring Severe Occlusion: Multi-Person 3D Pose Estimation with Gated Convolution.

Unsupervised Universal Hierarchical Multi-Person 3D Pose Estimation for Natural Scenes

HuMMan: Multi-Modal 4D Human Dataset for Versatile Sensing and Modeling

Fine-Grained Multi-human Parsing

Human-M3: A Multi-view Multi-modal Dataset for 3D Human Pose Estimation in Outdoor Scenes

LAMP: Leveraging Language Prompts for Multi-person Pose Estimation

LiCamPose: Combining Multi-View LiDAR and RGB Cameras for Robust Single-frame 3D Human Pose Estimation

Beyond Human Detection: A Benchmark for Detecting Common Human Posture

Direct Multi-view Multi-person 3D Pose Estimation

Joint Multi-Person Pose Estimation and Semantic Part Segmentation

Overcoming Data Deficiency for Multi-Person Pose Estimation

Single Person Pose Estimation: A Survey

FreeMan: Towards Benchmarking 3D Human Pose Estimation under Real-World Conditions

Human-Art: A Versatile Human-Centric Dataset Bridging Natural and Artificial Scenes

A Compact and Powerful Single-Stage Network for Multi-Person Pose Estimation

Shape and Pose Estimation for Closely Interacting Persons Using Multi-view Images.

HMOR: Hierarchical Multi-Person Ordinal Relations for Monocular Multi-Person 3D Pose Estimation

Learning 3-D Human Pose Estimation from Catadioptric Videos