Abstract:Learning an effective outfit-level representation is critical for predicting the compatibility of items in an outfit, and retrieving complementary items for a partial outfit. We present a framework, OutfitTransformer, that uses the proposed task-specific tokens and leverages the self-attention mechanism to learn effective outfit-level representations encoding the compatibility relationships between all items in the entire outfit for addressing both compatibility prediction and complementary item retrieval tasks. For compatibility prediction, we design an outfit token to capture a global outfit representation and train the framework using a classification loss. For complementary item retrieval, we design a target item token that additionally takes the target item specification (in the form of a category or text description) into consideration. We train our framework using a proposed set-wise outfit ranking loss to generate a target item embedding given an outfit, and a target item specification as inputs. The generated target item embedding is then used to retrieve compatible items that match the rest of the outfit. Additionally, we adopt a pre-training approach and a curriculum learning strategy to improve retrieval performance. Since our framework learns at an outfit-level, it allows us to learn a single embedding capturing higher-order relations among multiple items in the outfit more effectively than pairwise methods. Experiments demonstrate that our approach outperforms state-of-the-art methods on compatibility prediction, fill-in-the-blank, and complementary item retrieval tasks. We further validate the quality of our retrieval results with a user study.

COutfitGAN: Learning to Synthesize Compatible Outfits Supervised by Silhouette Masks and Fashion Styles

Learning to Synthesize Compatible Fashion Items Using Semantic Alignment and Collocation Classification: An Outfit Generation Framework

FCBoost-Net: A Generative Network for Synthesizing Multiple Collocated Outfits Via Fashion Compatibility Boosting

Fashion Recommendation on Street Images.

Collocated Clothing Synthesis with GANs Aided by Textual Information: A Multi-Modal Framework

BC-GAN: A Generative Adversarial Network for Synthesizing a Batch of Collocated Clothing

Outfit Recommendation with Deep Sequence Learning

Clothing Generation by Multi-Modal Embedding: A Compatibility Matrix-Regularized GAN Model.

Towards Intelligent Design: A Self-Driven Framework for Collocated Clothing Synthesis Leveraging Fashion Styles and Textures

Learning Outfit Compatibility with Graph Attention Network and Visual-Semantic Embedding.

Personalized Fashion Design

Complementary Factorization Towards Outfit Compatibility Modeling

Learning to Disentangle the Colors, Textures, and Shapes of Fashion Items: A Unified Framework

Fashion Compatibility Modeling through a Multi-modal Try-on-guided Scheme

Outfit compatibility model using fully connected self-adjusting graph neural network

Outfit Generation and Recommendation -- An Experimental Study

OutfitTransformer: Learning Outfit Representations for Fashion Recommendation

Transformer-based Graph Neural Networks for Outfit Generation

Toward AI fashion design: An Attribute-GAN model for clothing match

Learning compatibility knowledge for outfit recommendation with complementary clothing matching

MGCM: Multi-modal Generative Compatibility Modeling for Clothing Matching