iCONTRA: Toward Thematic Collection Design Via Interactive Concept Transfer

Dinh-Khoi Vo,Duy-Nam Ly,Khanh-Duy Le,Tam V. Nguyen,Minh-Triet Tran,Trung-Nghia Le

2024-03-14

Abstract:Creating thematic collections in industries demands innovative designs and cohesive concepts. Designers may face challenges in maintaining thematic consistency when drawing inspiration from existing objects, landscapes, or artifacts. While AI-powered graphic design tools offer help, they often fail to generate cohesive sets based on specific thematic concepts. In response, we introduce iCONTRA, an interactive CONcept TRAnsfer system. With a user-friendly interface, iCONTRA enables both experienced designers and novices to effortlessly explore creative design concepts and efficiently generate thematic collections. We also propose a zero-shot image editing algorithm, eliminating the need for fine-tuning models, which gradually integrates information from initial objects, ensuring consistency in the generation process without influencing the background. A pilot study suggests iCONTRA's potential to reduce designers' efforts. Experimental results demonstrate its effectiveness in producing consistent and high-quality object concept transfers. iCONTRA stands as a promising tool for innovation and creative exploration in thematic collection design. The source code will be available at:

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

### Problems the Paper Aims to Solve

The paper aims to address the challenges faced in the design industry when creating thematic collections. Specifically, designers may encounter difficulties in drawing inspiration from existing objects, landscapes, or artifacts to achieve thematic consistency in design collections. Although existing AI-based graphic design tools provide assistance, they often fail to generate cohesive sets of objects, particularly in maintaining specific thematic concepts.

### Main Contributions

1. **Proposing an Interactive Concept Transfer System (iCONTRA)**:
- iCONTRA is a user-friendly platform designed to help designers easily explore creative design concepts and efficiently generate thematically consistent object collections.
- The system allows users to upload original images and provide text descriptions in each generation unit to generate multiple objects. Users can further refine the generated objects through convenient import and edit prompt functions.

2. **Developing a Zero-Shot Image Editing Algorithm**:
- This algorithm achieves consistent and fine non-rigid object image generation without the need for model fine-tuning.
- By leveraging information from the original image, the algorithm prevents abrupt changes during the generation process and automatically masks foreground objects to reduce the impact of background adjustments.

3. **Preliminary Research Results**:
- Preliminary user research validated the effectiveness of iCONTRA in assisting users in generating relevant objects.
- Experimental results show that iCONTRA excels in transferring consistent and high-quality object concepts, demonstrating its potential in innovative design applications.

### Related Work

1. **Text-to-Image Generation**:
- Models based on diffusion techniques (such as GLIDE, DALL·E 2, and Imagen) have made significant progress in generating diverse, high-quality images aligned with complex text prompts.
- These models, through conditional generation methods, can generate target objects while keeping the background environment unchanged.

2. **Text-Based Image Editing**:
- Existing methods (such as DiffusionCLIP, Prompt-to-Prompt, and Masactrl) have made some progress in text-guided image editing but still face limitations, such as difficulty handling non-rigid transformations and information loss.
- iCONTRA's zero-shot image editing algorithm effectively addresses these issues by introducing a new attention mechanism, achieving more consistent and detailed non-rigid image synthesis and editing.

### System Implementation

1. **User Interface**:
- iCONTRA provides a web-based user interface, ensuring a simple and efficient user experience.
- Users can upload original object images and provide text descriptions in the six cells on the right side to generate multiple objects. The generated objects are guaranteed to be based on the original image and the user's input prompts.

2. **Zero-Shot Image Editing Algorithm**:
- The algorithm is based on the Stable Diffusion model, controlling the progressive fusion of original object content by gradually interpolating intermediate features, thus editing objects without changing the background.

### Preliminary Research

1. **Participants**:
- 4 students from a design club were invited to participate in the study, with 3 familiar with AI tools and 1 frequently using the Midjourney tool.

2. **Baseline Conditions and Measurements**:
- BingAI was chosen as the baseline tool for comparison, and the number of prompts required by users to generate the desired objects under both conditions was recorded.

3. **Tasks**:
- Five tasks were designed, including generating rings related to a given watch, motorcycles related to a given car, etc.

4. **Quantitative Results and Qualitative Feedback**:
- Participants found it difficult to find designs that met their expectations when using BingAI, while satisfaction was higher when using iCONTRA.
- User feedback indicated that iCONTRA performed well in reducing cognitive effort and generating the desired objects, but further improvements are needed, such as adding the ability to select specific areas for editing.

### Conclusion and Future Work

iCONTRA shows great potential in assisting designers in generating thematically consistent object collections. Future work will focus on further enhancing the system's capabilities and user experience.