Open-Endedness is Essential for Artificial Superhuman Intelligence

Edward Hughes,Michael Dennis,Jack Parker-Holder,Feryal Behbahani,Aditi Mavalankar,Yuge Shi,Tom Schaul,Tim Rocktaschel

2024-06-07

Abstract:In recent years there has been a tremendous surge in the general capabilities of AI systems, mainly fuelled by training foundation models on internetscale data. Nevertheless, the creation of openended, ever self-improving AI remains elusive. In this position paper, we argue that the ingredients are now in place to achieve openendedness in AI systems with respect to a human observer. Furthermore, we claim that such open-endedness is an essential property of any artificial superhuman intelligence (ASI). We begin by providing a concrete formal definition of open-endedness through the lens of novelty and learnability. We then illustrate a path towards ASI via open-ended systems built on top of foundation models, capable of making novel, humanrelevant discoveries. We conclude by examining the safety implications of generally-capable openended AI. We expect that open-ended foundation models will prove to be an increasingly fertile and safety-critical area of research in the near future.

Machine Learning,Artificial Intelligence

What problem does this paper attempt to address?

This paper explores the issue of openness in artificial intelligence (AI) systems, particularly the importance of achieving artificial superintelligence (ASI) that surpasses human capabilities. The authors point out that while the capabilities of AI are constantly improving, creating a self-improving and continuously innovative system remains a challenge. They propose that openness is a key attribute of any ASI, as it enables the generation of new knowledge and technology in a way similar to humans.

The paper first formalizes the definition of openness through novelty and learnability, and suggests that current AI systems based on large-scale data training, such as large language models, are powerful but lack true openness. The authors argue that future AI systems should be able to create, verify, and refine their own knowledge rather than just learning from a large amount of data.

The paper also discusses the safety concerns of open AI, emphasizing that this capability should be guided to benefit humanity and develop comprehensible knowledge. The authors propose that combining open basic models with open algorithms can efficiently discover new knowledge relevant to humans and guide the search space to ensure meaningful innovation in AI.

Finally, the paper presents several research directions, including combining openness and basic models through evolutionary algorithms and reinforcement learning. The authors highlight the importance of studying open systems for the secure and beneficial deployment of increasingly powerful and autonomous AI. They provide a formal definition of openness and compare it with the limitations of existing AI systems to illustrate the necessity of openness for ASI.