Abstract:Merchandise categories inherently form a semantic hierarchy with different levels of concept abstraction, especially for fine-grained categories. This hierarchy encodes rich correlations among various categories across different levels, which can effectively regularize the semantic space and thus make predictions less ambiguous. However, previous studies of fine-grained image retrieval primarily focus on semantic similarities or visual similarities. In a real application, merely using visual similarity may not satisfy the need of consumers to search merchandise with real-life images, e.g., given a red coat as a query image, we might get a red suit in recall results only based on visual similarity since they are visually similar. But the users actually want a coat rather than suit even the coat is with different color or texture attributes. We introduce this new problem based on photoshopping in real practice. That's why semantic information are integrated to regularize the margins to make "semantic" prior to "visual". To solve this new problem, we propose a hierarchical adaptive semantic-visual tree (ASVT) to depict the architecture of merchandise categories, which evaluates semantic similarities between different semantic levels and visual similarities within the same semantic class simultaneously. The semantic information satisfies the demand of consumers for similar merchandise with the query while the visual information optimizes the correlations within the semantic class. At each level, we set different margins based on the semantic hierarchy and incorporate them as prior information to learn a fine-grained feature embedding. To evaluate our framework, we propose a new dataset named JDProduct, with hierarchical labels collected from actual image queries and official merchandise images on an online shopping application. Extensive experimental results on the public CARS196 and CUB-

Structure-Aware Deep Learning for Product Image Classification

Fine Classification Method of Product Image Based on Multi-Level Convolutional Neural Networks

Automatic Fast Classification of Product-Images with Class-Specific Descriptor

Automatic Product Images Classification With Two Layers of SVM Classifier

Product Images Classification with Mul-tiple Features Combination

Large-Scale Product Classification via Spatial Attention Based CNN Learning and Multi-class Regression

Large Scale Product Categorization using Structured and Unstructured Attributes

Matryoshka Peek: Toward Learning Fine-Grained, Robust, Discriminative Features for Product Search

Is a picture worth a thousand words? A Deep Multi-Modal Fusion Architecture for Product Classification in e-commerce

Fine-Grained Grocery Product Recognition by One-Shot Learning.

Joint Hierarchical Category Structure Learning and Large-Scale Image Classification

Adaptive Semantic-Visual Tree for Hierarchical Embeddings

Object and attribute recognition for product image with self-supervised learning

Hierarchical Bilinear Convolutional Neural Network for Image Classification.

HCAM-CL: A Novel Method Integrating a Hierarchical Cross-Attention Mechanism with CNN-LSTM for Hierarchical Image Classification

Product Classification With the Motivation of Target Consumers by Deep Learning

DeepBag: Recognizing Handbag Models

A Unified Model with Structured Output for Fashion Images Classification

An Improved Deep Learning Approach For Product Recognition on Racks in Retail Stores

Learning Structured Inference Neural Networks with Label Relations

Multi-Object Classification for Deep Learning Analysis in CNN and MLP