Sparks of Artificial General Intelligence: Early experiments with GPT-4

Sébastien Bubeck,Varun Chandrasekaran,Ronen Eldan,Johannes Gehrke,Eric Horvitz,Ece Kamar,Peter Lee,Yin Tat Lee,Yuanzhi Li,Scott Lundberg,Harsha Nori,Hamid Palangi,Marco Tulio Ribeiro,Yi Zhang

2023-04-14

Abstract:Artificial intelligence (AI) researchers have been developing and refining large language models (LLMs) that exhibit remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition. The latest model developed by OpenAI, GPT-4, was trained using an unprecedented scale of compute and data. In this paper, we report on our investigation of an early version of GPT-4, when it was still in active development by OpenAI. We contend that (this early version of) GPT-4 is part of a new cohort of LLMs (along with ChatGPT and Google's PaLM for example) that exhibit more general intelligence than previous AI models. We discuss the rising capabilities and implications of these models. We demonstrate that, beyond its mastery of language, GPT-4 can solve novel and difficult tasks that span mathematics, coding, vision, medicine, law, psychology and more, without needing any special prompting. Moreover, in all of these tasks, GPT-4's performance is strikingly close to human-level performance, and often vastly surpasses prior models such as ChatGPT. Given the breadth and depth of GPT-4's capabilities, we believe that it could reasonably be viewed as an early (yet still incomplete) version of an artificial general intelligence (AGI) system. In our exploration of GPT-4, we put special emphasis on discovering its limitations, and we discuss the challenges ahead for advancing towards deeper and more comprehensive versions of AGI, including the possible need for pursuing a new paradigm that moves beyond next-word prediction. We conclude with reflections on societal influences of the recent technological leap and future research directions.

Computation and Language,Artificial Intelligence

What problem does this paper attempt to address?

The paper primarily explores the potential and progress of early versions of GPT-4 (developed by OpenAI) in demonstrating Artificial General Intelligence (AGI). The paper points out that GPT-4, as a Large Language Model (LLM), not only excels in language mastery but also displays impressive capabilities in a variety of non-linguistic tasks, including mathematics, programming, visual understanding, and music composition, with performance nearing or surpassing human levels.

The authors of the paper demonstrate the model's flexibility in handling complex and innovative tasks through interactive experiments with GPT-4, such as proving the infiniteness of prime numbers in poetic form and drawing unicorn images with code. These tasks require the model to have the ability to integrate knowledge across domains and creative thinking. In addition, GPT-4 can understand and correct code, as well as infer and complete visual features in the absence of information, indicating that it possesses a certain degree of "visual" understanding capability, despite being fundamentally a pure text model.

The paper emphasizes that GPT-4's abilities go beyond mere pattern memorization and imitation, showing a profound understanding of concepts, skills, and domains. However, GPT-4 is not without its imperfections; it still has some limitations and biases, especially in planning and long-term goal setting. The paper also discusses the challenges of GPT-4's social impact, including the generation of errors, the spread of misleading information, potential occupational impacts, and economic consequences.

In summary, although GPT-4 approaches or achieves human intelligence levels in some respects, its mode of intelligence is distinctly different from that of humans. The paper suggests that GPT-4 represents an important step towards a more broadly defined AGI, even though there are many areas in need of improvement. Finally, the paper calls for further research to better understand and assess artificial intelligence systems with broader intelligence capabilities.