Abstract:Understanding humans from photographs has always been a fundamental goal of computer vision. In this thesis we have developed a hierarchy of tools that cover a wide range of topics with the objective of understanding humans from monocular RGB image: from low level feature point descriptors to high level fashion-aware conditional random fields models. In order to build these high level models it is paramount to have a battery of robust and reliable low and mid level cues. Along these lines, we have proposed two low-level keypoint descriptors: one based on the theory of the heat diffusion on images, and the other that uses a convolutional neural network to learn discriminative image patch representations. We also introduce distinct low-level generative models for representing human pose: in particular we present a discrete model based on a directed acyclic graph and a continuous model that consists of poses clustered on a Riemannian manifold. As mid level cues we propose two 3D human pose estimation algorithms: one that estimates the 3D pose given a noisy 2D estimation, and an approach that simultaneously estimates both the 2D and 3D pose. Finally, we formulate higher level models built upon low and mid level cues for understanding humans from single images. Concretely, we focus on two different tasks in the context of fashion: semantic segmentation of clothing, and predicting the fashionability from images with metadata to ultimately provide fashion advice to the user. For all presented approaches we present extensive results and comparisons against the state-of-the-art and show significant improvements on the entire variety of tasks we tackle.

Human-Centric Image Categorization Based on Poselets

Human-Centric Image Captioning

A Pca Based Automatic Image Categorization Approach Using Dominant Color Features

Capturing human categorization of natural images at scale by combining deep networks and cognitive models

Human Action Recognition with Contextual Constraints Using a RGB-D Sensor

Partially Occluded Head Posture Estimation for 2D Images Using Pyramid HoG Features

Understanding Human-Centric Images: From Geometry to Fashion

Human-Art: A Versatile Human-Centric Dataset Bridging Natural and Artificial Scenes

Human Pose as Compositional Tokens

Pose2Seg: Detection Free Human Instance Segmentation

Human Pose Descriptions and Subject-Focused Attention for Improved Zero-Shot Transfer in Human-Centric Classification Tasks

Pose for Everything: Towards Category-Agnostic Pose Estimation

Human-Centric Images and Videos Analysis

UniHCP: A Unified Model for Human-Centric Perceptions

HS-Pose: Hybrid Scope Feature Extraction for Category-level Object Pose Estimation

Pose2Seg: Human Instance Segmentation Without Detection.

3d Pose Detection Of Closely Interactive Humans Using Multi-View Cameras

A Survey on Monocular 3D Human Pose Estimation

Action recognition in still images using a combination of human pose and context information

Visual saliency coding for image categorization

Human Pose Estimation in Monocular Omnidirectional Top-View Images