Dressing the Imagination: A Dataset for AI-Powered Translation of Text into Fashion Outfits and A Novel KAN Adapter for Enhanced Feature Adaptation

Gayatri Deshmukh,Somsubhra De,Chirag Sehgal,Jishu Sen Gupta,Sparsh Mittal
2024-11-21
Abstract:Specialized datasets that capture the fashion industry's rich language and styling elements can boost progress in AI-driven fashion design. We present FLORA (Fashion Language Outfit Representation for Apparel Generation), the first comprehensive dataset containing 4,330 curated pairs of fashion outfits and corresponding textual descriptions. Each description utilizes industry-specific terminology and jargon commonly used by professional fashion designers, providing precise and detailed insights into the outfits. Hence, the dataset captures the delicate features and subtle stylistic elements necessary to create high-fidelity fashion designs. We demonstrate that fine-tuning generative models on the FLORA dataset significantly enhances their capability to generate accurate and stylistically rich images from textual descriptions of fashion sketches. FLORA will catalyze the creation of advanced AI models capable of comprehending and producing subtle, stylistically rich fashion designs. It will also help fashion designers and end-users to bring their ideas to life. As a second orthogonal contribution, we introduce KAN Adapters, which leverage Kolmogorov-Arnold Networks (KAN) as adaptive modules. They serve as replacements for traditional MLP-based LoRA adapters. With learnable spline-based activations, KAN Adapters excel in modeling complex, non-linear relationships, achieving superior fidelity, faster convergence and semantic alignment. Extensive experiments and ablation studies on our proposed FLORA dataset validate the superiority of KAN Adapters over LoRA adapters. To foster further research and collaboration, we will open-source both the FLORA and our implementation code.
Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
The main problem that this paper attempts to solve is to generate high - quality fashion clothing sketches. Specifically, the paper makes two main contributions: 1. **FLORA Dataset**: - **Problem Background**: Existing datasets are insufficient in generating high - quality fashion clothing sketches, especially lacking pairs of detailed annotated text descriptions and corresponding sketches. - **Solution**: The paper introduces the FLORA dataset, which is a dataset containing 4,330 pairs of fashion clothing sketches and their detailed text descriptions. Each text description uses industry - specific terms to capture the subtle elements in the design, enabling the generation of high - fidelity fashion clothing sketches. - **Objective**: By providing a high - quality dataset, it helps train the generation model so that it can generate accurate and visually appealing fashion clothing sketches from text descriptions, reducing the time designers spend on initial sketching and improving the efficiency of the design process. 2. **KAN Adapter**: - **Problem Background**: Existing models have limitations in handling complex, non - linear data patterns, especially when generating highly detailed and rich - style fashion sketches. - **Solution**: The paper proposes a new KAN Adapter, which uses Kolmogorov - Arnold Networks (KAN) as an adaptive module to replace the traditional MLP - based LoRA adapter. The KAN Adapter can model complex non - linear relationships more effectively through learnable spline activation functions, achieving higher fidelity, faster convergence speed and better semantic alignment. - **Objective**: By improving the adaptability and expressiveness of the model, it enhances the performance of the generation model in generating fashion clothing sketches, especially when dealing with fine - grained details and multi - modal alignment. ### Summary of Main Contributions - **FLORA Dataset**: Provides a large - scale, carefully curated dataset, filling the gap in existing resources in the field of text - to - fashion - sketch generation. - **KAN Adapter**: Proposes a new adapter architecture, which enhances the non - linear modeling ability of the model through the KAN network, improving the generation quality and training efficiency. These contributions aim to promote the application of AI in the field of fashion design, reduce the dependence on traditional hand - drawing skills, accelerate the design process, and enhance the creativity and quality of the final product.