Abstract:The ability to correctly classify and retrieve apparel images has a variety of applications important to e-commerce, online advertising and internet search. In this work, we propose a robust framework for fine-grained apparel classification, in-shop and cross-domain retrieval which eliminates the requirement of rich annotations like bounding boxes and human-joints or clothing landmarks, and training of bounding box/ key-landmark detector for the same. Factors such as subtle appearance differences, variations in human poses, different shooting angles, apparel deformations, and self-occlusion add to the challenges in classification and retrieval of apparel items. Cross-domain retrieval is even harder due to the presence of large variation between online shopping images, usually taken in ideal lighting, pose, positive angle and clean background as compared with street photos captured by users in complicated conditions with poor lighting and cluttered scenes. Our framework uses compact bilinear CNN with tensor sketch algorithm to generate embeddings that capture local pairwise feature interactions in a translationally invariant manner. For apparel classification, we pass the feature embeddings through a softmax classifier, while, the in-shop and cross-domain retrieval pipelines use a triplet-loss based optimization approach, such that squared Euclidean distance between embeddings measures the dissimilarity between the images. Unlike previous works that relied on bounding box, key clothing landmarks or human joint detectors to assist the final deep classifier, proposed framework can be trained directly on the provided category labels or generated triplets for triplet loss optimization. Lastly, Experimental results on the DeepFashion fine-grained categorization, and in-shop and consumer-to-shop retrieval datasets provide a comparative analysis with previous work performed in the domain.

An End-to-End Framework for Clothing Collocation Based on Semantic Feature Fusion

Deforming Garment Classification with Shallow Temporal Extraction and Tree-Based Fusion

Feature fusion network for clothing parsing

MMFL-Net: Multi-scale and Multi-granularity Feature Learning for Cross-domain Fashion Retrieval

A Part-Based and Feature Fusion Method for Clothing Classification.

Outfit compatibility prediction with multi-layered feature fusion network

Multideep Feature Fusion Algorithm for Clothing Style Recognition

Learning to Match Clothing from Textual Feature-Based Compatible Relationships

A Fine-Grained Vision and Language Representation Framework with Graph-Based Fashion Semantic Knowledge

Towards Better Understanding the Clothing Fashion Styles: A Multimodal Deep Learning Approach

Fine-grained Apparel Classification and Retrieval without rich annotations

Clothes Co-Parsing Via Joint Image Segmentation and Labeling with Application to Clothing Retrieval

Multiple Features with Extreme Learning Machines for Clothing Image Recognition

Learning Visual Clothing Style with Heterogeneous Dyadic Co-occurrences

Clothes Collocation Recommendations by Compatibility Learning

Learning to Collocate Fashion Items from Heterogeneous Network Using Structural and Textual Features

Outfit Recommendation with Deep Sequence Learning

FFFN: Fashion Feature Fusion Network by Co-Attention Model for Fashion Recommendation

Learning Fashion Compatibility with Bidirectional LSTMs

Clothing retrieval combining hierarchical over-segmentation and cross-domain dictionary learning

Clothing Co-Parsing by Joint Image Segmentation and Labeling