Semantic Draw Engineering for Text-to-Image Creation

Yang Li,Huaqiang Jiang,Yangkai Wu

2023-12-23

Abstract:Text-to-image generation is conducted through Generative Adversarial Networks (GANs) or transformer models. However, the current challenge lies in accurately generating images based on textual descriptions, especially in scenarios where the content and theme of the target image are ambiguous. In this paper, we propose a method that utilizes artificial intelligence models for thematic creativity, followed by a classification modeling of the actual painting process. The method involves converting all visual elements into quantifiable data structures before creating images. We evaluate the effectiveness of this approach in terms of semantic accuracy, image reproducibility, and computational efficiency, in comparison with existing image generation algorithms.

Human-Computer Interaction,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper attempts to address the challenge of generating high-quality images in current text-to-image generation technology, especially when the text descriptions are vague in content and theme. Specifically, the paper focuses on the following aspects:

1. **Semantic Complexity and Ambiguity**: The complexity and ambiguity of natural language descriptions make it difficult for most users (especially those unfamiliar with the model) to generate effective prompts, thereby affecting the quality of the final generated images.

2. **Personalized Needs**: Existing methods are too generic and fail to meet users' personalized image and precise creative needs.

3. **Parameter Adjustment and Quality Evaluation**: Different models have different parameter settings, and there is a lack of a systematic approach to generate high-quality results and evaluate the quality of the prompts.

To address these issues, the paper proposes a method based on Semantic Draw Engineering (SDE), which breaks down the artistic creation process into 6 steps: ideation, theme conceptualization, sketch outlining, content expression, light and shadow processing, and refinement. Through this method, the paper aims to improve the controllability, accuracy, and precision of generated images, especially for users unfamiliar with artistic knowledge, such as researchers, helping them quickly generate high-quality illustrations based on paper abstracts.