A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT

Yihan Cao,Siyu Li,Yixin Liu,Zhiling Yan,Yutong Dai,Philip S. Yu,Lichao Sun

2023-03-08

Abstract:Recently, ChatGPT, along with DALL-E-2 and Codex,has been gaining significant attention from society. As a result, many individuals have become interested in related resources and are seeking to uncover the background and secrets behind its impressive performance. In fact, ChatGPT and other Generative AI (GAI) techniques belong to the category of Artificial Intelligence Generated Content (AIGC), which involves the creation of digital content, such as images, music, and natural language, through AI models. The goal of AIGC is to make the content creation process more efficient and accessible, allowing for the production of high-quality content at a faster pace. AIGC is achieved by extracting and understanding intent information from instructions provided by human, and generating the content according to its knowledge and the intent information. In recent years, large-scale models have become increasingly important in AIGC as they provide better intent extraction and thus, improved generation results. With the growth of data and the size of the models, the distribution that the model can learn becomes more comprehensive and closer to reality, leading to more realistic and high-quality content generation. This survey provides a comprehensive review on the history of generative models, and basic components, recent advances in AIGC from unimodal interaction and multimodal interaction. From the perspective of unimodality, we introduce the generation tasks and relative models of text and image. From the perspective of multimodality, we introduce the cross-application between the modalities mentioned above. Finally, we discuss the existing open problems and future challenges in AIGC.

Artificial Intelligence,Computation and Language,Machine Learning

What problem does this paper attempt to address?

The paper primarily explores the development and current status of Artificial Intelligence Generated Content (AIGC), particularly the technological evolution from Generative Adversarial Networks (GAN) to ChatGPT. Below is a summary of the core issues the paper attempts to address:

1. **Definition and Review of AIGC**:
- Provide a formal definition of AIGC and a comprehensive review of AIGC and its enhanced generation process.
- AIGC refers to content automatically generated by advanced generative AI technologies, such as text, images, and music.

2. **Historical Review and Technical Foundation**:
- Review the historical development of AIGC, including the development of Natural Language Processing (NLP), Computer Vision (CV), and Visual Language (VL) modalities.
- Discuss the key technologies underpinning AIGC, such as the Transformer architecture, pre-trained language models, reinforcement learning, and advancements in computational resources.

3. **Recent Advances**:
- Summarize recent advances in unimodal and multimodal generative models.
- Unimodal generative models focus on specific types of data, such as text or images, while multimodal models can handle instructions across different data types.
- Introduce the application of models like the GPT series, DALL-E series, and other Transformer-based models in generative tasks.

4. **Challenges and Future Directions**:
- Explore the main challenges in the AIGC field, such as better understanding human intentions and improving the quality of generated content.
- Point out future research trends and development directions, including more efficient model training methods, new evaluation metrics, and considerations of ethical and social impacts.

In summary, this paper aims to provide a comprehensive overview of AIGC, covering its historical background, key technologies, recent advances, challenges, and future research directions. This is of significant importance for understanding and advancing the development of generative AI technologies.