Abstract:In this paper, we present effective algorithms to automatically annotate clothes from social media data, such as Facebook and Instagram. Clothing annotation can be informally stated as recognizing, as accurately as possible, the garment items appearing in the query photo. This task brings huge opportunities for recommender and e-commerce systems, such as capturing new fashion trends based on which clothes have been used more recently. It also poses interesting challenges for existing vision and recognition algorithms, such as distinguishing between similar but different types of clothes or identifying a pattern of a cloth even if it has different colors and shapes. We formulate the annotation task as a multi-label and multi-modal classification problem: (i) both image and textual content (i.e., tags about the image) are available for learning classifiers, (ii) the classifiers must recognize a set of labels (i.e., a set of garment items), and (iii) the decision on which labels to assign to the query photo comes from a set of instances that is used to build a function, which separates labels that should be assigned to the query photo, from those that should not be assigned. Using this configuration, we propose two approaches: (i) the pointwise one, called MMCA, which receives a single image as input, and (ii) a multi-instance classification, called M3CA, also known as pairwise approach, which uses pair of images to create the classifiers. We conducted a systematic evaluation of the proposed algorithms using everyday photos collected from two major fashion-related social media, namely pose.com and chictopia.com. Our results show that the proposed approaches provide improvements when compared to popular first choice multi-label, multi-modal, multi-instance algorithms that range from 20 % to 30 % in terms of accuracy.

Garment Detectives: Discovering Clothes and Its Genre in Consumer Photos

Learning and Recognition of Clothing Genres From Full-Body Images

Automatic Measurement of Garment Sizes Using Image Recognition

Fashion Recommendation on Street Images.

Cross-Scenario Clothing Retrieval And Fine-Grained Style Recognition

Fashion Meets Computer Vision

From Culture to Clothing: Discovering the World Events Behind A Century of Fashion Images

Searching for Apparel Products from Images in the Wild

Clothing Co-Parsing by Joint Image Segmentation and Labeling

Fashion Meets Computer Vision: A Survey

An Approach for Clothing Recommendation Based on Multiple Image Attributes

Learning To Appreciate The Aesthetic Effects Of Clothing

Finding suits in images of people

Snap and Find: Deep Discrete Cross-domain Garment Image Retrieval

StreetStyle: Exploring world-wide clothing styles from millions of photos

Pointwise and pairwise clothing annotation: combining features from social media

Two-Stage Method for Clothing Feature Detection

Garment Style Creator: Using StarGAN for Image-to-Image Translation of Multi-Domain Garments

When Fashion Meets Big Data: Discriminative Mining of Best Selling Clothing Features

GeoStyle: Discovering Fashion Trends and Events

Detailed Garment Recovery from a Single-View Image