Examining the Differential Risk from High-level Artificial Intelligence and the Question of Control

Kyle A. Kilian,Christopher J. Ventura,Mark M. Bailey
DOI: https://doi.org/10.1016/j.futures.2023.103182
2023-11-24
Abstract:Artificial Intelligence (AI) is one of the most transformative technologies of the 21st century. The extent and scope of future AI capabilities remain a key uncertainty, with widespread disagreement on timelines and potential impacts. As nations and technology companies race toward greater complexity and autonomy in AI systems, there are concerns over the extent of integration and oversight of opaque AI decision processes. This is especially true in the subfield of machine learning (ML), where systems learn to optimize objectives without human assistance. Objectives can be imperfectly specified or executed in an unexpected or potentially harmful way. This becomes more concerning as systems increase in power and autonomy, where an abrupt capability jump could result in unexpected shifts in power dynamics or even catastrophic failures. This study presents a hierarchical complex systems framework to model AI risk and provide a template for alternative futures analysis. Survey data were collected from domain experts in the public and private sectors to classify AI impact and likelihood. The results show increased uncertainty over the powerful AI agent scenario, confidence in multiagent environments, and increased concern over AI alignment failures and influence-seeking behavior.
Artificial Intelligence,Computers and Society,Human-Computer Interaction
What problem does this paper attempt to address?
### What Problem Does the Paper Attempt to Solve? This paper explores the risks and uncertainties associated with the development of advanced artificial intelligence (AI) systems, particularly artificial general intelligence (AGI), and analyzes these risks and potential future scenarios through the construction of a multi-level complex systems framework. Specifically, the paper focuses on the following aspects: 1. **Classification of AI System Risks**: - **Misuse Risks**: For example, cyber threat actors using AI technology for attacks or generating false information. - **Accident Risks**: Including unexpected failure modes, which may be the responsibility of the system or developers. - **Agent Risks**: Refers to AI agents capable of autonomous interaction and goal achievement (such as AlphaGo), especially when the goals of these agents are inconsistent with human goals. - **Structural Risks**: Involves complex interactions within social and technical systems that affect the development and deployment of AI technology. 2. **Technological Transformation and Competition Dynamics**: - Investigates the impact of the speed of technological progress, the extent of diffusion, and competitive relationships during the transformation process on AI risks. - Pays special attention to the potential disruptive effects of automated systems on the stability of strategic weapon systems and the resulting trust erosion issues. 3. **Control Measures**: - Explores how to address AI risks through technical safety, institutional safety, and governance strategies. - Analyzes existing technical limitations and interpretability barriers, which are difficult to solve through governance and regulation alone. 4. **Future Scenario Modeling**: - Proposes a novel exploratory modeling technique to characterize future scenarios and their associated risks through a multi-level complex systems framework. - Structures the key dimensions and conditions of AI risks, revealing highly variable future landscapes. In summary, the paper aims to comprehensively assess the potential risks of advanced AI development and proposes a systematic framework to guide future research and policy-making.