Abstract:The revolution of artificial intelligence content generation has been rapidly accelerated with the booming text-to-image (T2I) diffusion models. Within just two years of development, it was unprecedentedly of high-quality, diversity, and creativity that the state-of-the-art models could generate. However, a prevalent limitation persists in the effective communication with these popular T2I models, such as Stable Diffusion, using natural language descriptions. This typically makes an engaging image hard to obtain without expertise in prompt engineering with complex word compositions, magic tags, and annotations. Inspired by the recently released DALLE3 - a T2I model directly built-in ChatGPT that talks human language, we revisit the existing T2I systems endeavoring to align human intent and introduce a new task - interactive text to image (iT2I), where people can interact with LLM for interleaved high-quality image generation/edit/refinement and question answering with stronger images and text correspondences using natural language. In addressing the iT2I problem, we present a simple approach that augments LLMs for iT2I with prompting techniques and off-the-shelf T2I models. We evaluate our approach for iT2I in a variety of common-used scenarios under different LLMs, e.g., ChatGPT, LLAMA, Baichuan, and InternLM. We demonstrate that our approach could be a convenient and low-cost way to introduce the iT2I ability for any existing LLMs and any text-to-image models without any training while bringing little degradation on LLMs' inherent capabilities in, e.g., question answering and code generation. We hope this work could draw broader attention and provide inspiration for boosting user experience in human-machine interactions alongside the image quality of the next-generation T2I systems.

Harnessing the Power of Images in CALL: AI Image Generation for Context Specific Visual Aids in Less Commonly Taught Languages

Learn, Imagine and Create: Text-to-Image Generation from Prior Knowledge.

AI-empowered Imagery Writing: Integrating AI-generated Imagery into Digital Mental Health Service

The implementation of the cognitive theory of multimedia learning in the design and evaluation of an AI educational video assistant utilizing large language models

GenAssist: Making Image Generation Accessible

Using Call Annie as a Generative Artificial Intelligence Speaking Partner for Language Learners

Mini-DALLE3: Interactive Text to Image by Prompting Large Language Models

New Trends in Second Language Learning and Teaching through the lens of ICT, Networked Learning, and Artificial Intelligence

Developing an AI-Assisted Low-Resource Spoken Language Learning App for Children

LLM as an Art Director (LaDi): Using LLMs to improve Text-to-Media Generators

ContextCam: Bridging Context Awareness with Creative Human-AI Image Co-Creation

Interactive Data Synthesis for Systematic Vision Adaptation via LLMs-AIGCs Collaboration

Text to Image Generation: Leaving no Language Behind

Preparing teachers for the application of AI-powered technologies in foreign language education

RetAssist: Facilitating Vocabulary Learners with Generative Images in Story Retelling Practices

AI Illustrator: Translating Raw Descriptions into Images by Prompt-based Cross-Modal Generation

Enriching Education with Artificial Intelligence Generative Ai-Speech to Image Generator

Elucidating the design space of language models for image generation

An image speaks a thousand words, but can everyone listen? On image transcreation for cultural relevance

Empowering Visual Creativity: A Vision-Language Assistant to Image Editing Recommendations

Exploring the use of Generative AI to Support Automated Just-in-Time Programming for Visual Scene Displays