Abstract:The ability to correctly classify and retrieve apparel images has a variety of applications important to e-commerce, online advertising and internet search. In this work, we propose a robust framework for fine-grained apparel classification, in-shop and cross-domain retrieval which eliminates the requirement of rich annotations like bounding boxes and human-joints or clothing landmarks, and training of bounding box/ key-landmark detector for the same. Factors such as subtle appearance differences, variations in human poses, different shooting angles, apparel deformations, and self-occlusion add to the challenges in classification and retrieval of apparel items. Cross-domain retrieval is even harder due to the presence of large variation between online shopping images, usually taken in ideal lighting, pose, positive angle and clean background as compared with street photos captured by users in complicated conditions with poor lighting and cluttered scenes. Our framework uses compact bilinear CNN with tensor sketch algorithm to generate embeddings that capture local pairwise feature interactions in a translationally invariant manner. For apparel classification, we pass the feature embeddings through a softmax classifier, while, the in-shop and cross-domain retrieval pipelines use a triplet-loss based optimization approach, such that squared Euclidean distance between embeddings measures the dissimilarity between the images. Unlike previous works that relied on bounding box, key clothing landmarks or human joint detectors to assist the final deep classifier, proposed framework can be trained directly on the provided category labels or generated triplets for triplet loss optimization. Lastly, Experimental results on the DeepFashion fine-grained categorization, and in-shop and consumer-to-shop retrieval datasets provide a comparative analysis with previous work performed in the domain.

Scaling Cross-Domain Content-Based Image Retrieval for E-commerce Snap and Search Application

Visual Search at eBay

Visual Search at Alibaba

CSIR4G: an Effective and Efficient Cross-Scenario Image Retrieval Model for Glasses

MSURU: Large Scale E-commerce Image Classification with Weakly Supervised Search Data

Fast Object Retrieval Using Direct Spatial Matching

What Image Do You Need? A Two-stage Framework for Image Selection in E-commerce.

Efficient Indexing for Large-Scale Image Search

Images Don't Lie: Transferring Deep Visual Semantic Features to Large-Scale Multimodal Learning to Rank

Cross-domain Image Retrieval with a Dual Attribute-aware Ranking Network

Deep Learning based Large Scale Visual Recommendation and Search for E-Commerce

From Document to Image: Learning a Scalable Ranking Model for Content Based Image Retrieval

Multi-Scale Visual Words For Object-Based Web Image Search

Cascade Category-Aware Visual Search

Fine-grained Apparel Classification and Retrieval without rich annotations

Large-Scale Product Retrieval with Weakly Supervised Representation Learning

Cross-Domain Product Representation Learning for Rich-Content E-Commerce

Cross-Domain Image Retrieval with Attention Modeling

Open Cross-Domain Visual Search

Extending CLIP for Category-to-image Retrieval in E-commerce

Real20M: A Large-scale E-commerce Dataset for Cross-domain Retrieval