Abstract:This paper considers the problem of text-based person search, which aims to find the target person based on a query textual description. Previous methods commonly focus on learning shared image-text embeddings, but largely ignore the effect of pedestrian attributes. Attributes are fine-grained information, which provide mid-level semantics and have been demonstrated to be effective in traditional image-based person search. However, in text-based person search, it is hard to incorporate attribute information to learn discriminative image-text embeddings, because (1) the description of attributes could be various at different texts and (2) it is hard to decouple attributes-related information without the help of attribute annotations. In this paper, we propose an improving embedding learning by virtual attribute decoupling (iVAD) model for learning modality-invariant image-text embeddings. To the best of our knowledge, this is the first work which performs unsupervised attribute decoupling in text-based person search task. In the iVAD, we first propose a novel virtual attribute decoupling (VAD) module which uses an encoder-decoder embedding learning structure to decompose attribute information from image and text. In this module, we regard the pedestrian attributes as a hidden vector and obtain attribute-related embeddings. In addition, different from previous works which separates attribute learning from image-text embedding learning, we propose a hierarchical feature embedding framework. We incorporate the attribute-related embeddings into learned image-text embeddings by an attribute-enhanced feature embedding (AEFE) module. The proposed AEFE module can utilize attribute information to improve discriminability of learned features. Extensive evaluations demonstrate the superiority of our method over a wide variety of state-of-the-art methods on the CUHK-PEDES dataset. The experimental results on Caltech-UCSD Birds (CUB), Oxford-102 Flowers (Flowers) and Flickr30K verify the effectiveness of the proposed approach. A further visualization shows that the proposed iVAD model can effectively discover the co-occurring pedestrian attributes in corresponded image-text pairs.

Feature Decomposition and Attribute Augmentation for Attribute-based Person Search.

Model Semantic Relations with Extended Attributes

Learning Weak Semantics by Feature Graph for Attribute-Based Person Search

Multi-Attribute Enhancement Network for Person Search

Improving Embedding Learning by Virtual Attribute Decoupling for Text-Based Person Search

Comprehensive Attribute Prediction Learning for Person Search by Language.

Improving Inconspicuous Attributes Modeling for Person Search by Language

Attribute-Aware Feature Encoding for Object Recognition and Segmentation

Improving Cross-Modal Constraints: Text Attribute Person Search with Graph Attention Networks

Improving Pedestrian Attribute Recognition With Weakly-Supervised Multi-Scale Attribute-Specific Localization

Cross-modal Co-occurrence Attributes Alignments for Person Search by Language

Adversarial Attribute-Text Embedding for Person Search with Natural Language Query

Attribute-Aligned Domain-Invariant Feature Learning For Unsupervised Domain Adaptation Person Re-Identification

A novel self-boosting dual-branch model for pedestrian attribute recognition

Beyond the Parts: Learning Coarse-to-Fine Adaptive Alignment Representation for Person Search

ARM Feature Embedding + Feature Extractor Feature Aggregation MLP ... Pool zzii / wwbbii Co-occurrence ( a ) ( b ) Attribute Prototype Embedding ( APE ) Classifier

Text-based Person Search Via Virtual Attribute Learning

Person Retrieval in Surveillance Videos Via Deep Attribute Mining and Reasoning

3 a : a person re-identification system via attribute augmentation and aggregation

Unleashing the Feature Hierachy Potential: an Efficient Tri-Hybrid Person Search Model

AttrSeg: Open-Vocabulary Semantic Segmentation via Attribute Decomposition-Aggregation