Regulating advanced artificial agents

Michael K Cohen,Noam Kolt,Yoshua Bengio,Gillian K Hadfield,Stuart Russell,Michael K. Cohen,Gillian K. Hadfield

DOI: https://doi.org/10.1126/science.adl0625

IF: 56.9

2024-04-05

Science

Abstract:Governance frameworks should address the prospect of AI systems that cannot be safely tested

multidisciplinary sciences

What problem does this paper attempt to address?

Appraising Regulatory Framework Towards Artificial General Intelligence (AGI) under Digital Humanism

Le Cheng,Xuan Gong

DOI: https://doi.org/10.1515/ijdlg-2024-0015

2024-01-01

Abstract:Abstract The explosive advancement of contemporary artificial intelligence (AI) technologies, typified by ChatGPT, is steering humans towards an uncontrollable trajectory to artificial general intelligence (AGI). Against the backdrop of a series of transformative breakthroughs, big tech companies such as OpenAI and Google have initiated an “AGI race” on a supranational level. As technological power becomes increasingly absolute, structural challenges may erupt with an unprecedented velocity, potentially resulting in disorderly expansion and even malignant development of AI technologies. To preserve the dignity and safety of human-beings in a brand-new AGI epoch, it is imperative to implement regulatory guidelines to limit the applications of AGI within the confines of human ethics and rules to further counteract the potential downsides. To promote the benevolent evolution of AGI, the principles of Humanism should be underscored and the connotation of Digital Humanism should be further enriched. Correspondingly, the current regulatory paradigm for generative AI may also be overhauled under the tenet of Digital Humanism to adapt to the quantum leaps and subversive shifts produced by AGI in the future. Positioned at the nexus of legal studies, computer science, and moral philosophy, this study therefore charts a course for a synthetic regulation framework of AGI under Digital Humanism.
Artificially Intelligent Regulation

Mariano-Florentino Cuéllar,Aziz Z. Huq

DOI: https://doi.org/10.1162/daed_a_01920

2022-01-01

Daedalus

Abstract:Abstract This essay maps the potential, and risks, of artificially intelligent regulation: regulatory arrangements that use a complex computational algorithm or another artificial agent either to define a legal norm or to guide its implementation. The ubiquity of AI systems in modern organizations all but guarantees that regulators or the parties they regulate will make use of learning algorithms or novel techniques to analyze data in the process of defining, implementing, or complying with regulatory requirements. We offer an account of the possible benefits and harms of artificially intelligent regulation. Its mix of costs and rewards, we show, depend primarily on whether AI is deployed in ways aimed merely at shoring up existing hierarchies, or whether AI systems are embedded in and around legal frameworks carefully structured and evaluated to better our lives, environment, and future.

social sciences, interdisciplinary
Regulating artificial intelligence: Proposal for a global solution

Olivia J. Erdélyi,Judy Goldsmith

DOI: https://doi.org/10.1016/j.giq.2022.101748

IF: 8.49

2022-10-01

Government Information Quarterly

Abstract:With increasing ubiquity of artificial intelligence (AI) in modern societies, individual countries and the international community are working hard to create an innovation-friendly, yet safe, regulatory environment. Adequate regulation is key to maximize the benefits and minimize the risks stemming from AI technologies. Developing regulatory frameworks is, however, challenging due to AI's global reach, agency problems present in regulation, and the existence of widespread misconceptions about the very notion of regulation. This paper makes three claims: (1) Based on interdisciplinary insights, we show that AI-related challenges cannot be tackled effectively without sincere international coordination supported by robust, consistent domestic, regional, and international governance arrangements. (2) Against this backdrop, we propose the establishment of an international AI governance framework to spearhead initiatives to create a consistent, global enabling regulatory environment, which is necessary for the successful and responsible adoption of AI technologies. To facilitate the practical implementation of our recommendation, we provide a simplified impact assessment on regulatory architecture and governance design options, appropriate to the scope of the paper. (3) We draw attention to communication challenges, which we believe are underestimated barriers hindering contemporary efforts to develop AI regulatory regimes. We argue that a fundamental change of mindset regarding the nature of regulation is necessary to remove these, and put forward some recommendations on how to achieve this.

information science & library science
Frontier AI Regulation: Managing Emerging Risks to Public Safety

Markus Anderljung,Joslyn Barnhart,Anton Korinek,Jade Leung,Cullen O'Keefe,Jess Whittlestone,Shahar Avin,Miles Brundage,Justin Bullock,Duncan Cass-Beggs,Ben Chang,Tantum Collins,Tim Fist,Gillian Hadfield,Alan Hayes,Lewis Ho,Sara Hooker,Eric Horvitz,Noam Kolt,Jonas Schuett,Yonadav Shavit,Divya Siddarth,Robert Trager,Kevin Wolf

2023-11-08

Abstract:Advanced AI models hold the promise of tremendous benefits for humanity, but society needs to proactively manage the accompanying risks. In this paper, we focus on what we term "frontier AI" models: highly capable foundation models that could possess dangerous capabilities sufficient to pose severe risks to public safety. Frontier AI models pose a distinct regulatory challenge: dangerous capabilities can arise unexpectedly; it is difficult to robustly prevent a deployed model from being misused; and, it is difficult to stop a model's capabilities from proliferating broadly. To address these challenges, at least three building blocks for the regulation of frontier models are needed: (1) standard-setting processes to identify appropriate requirements for frontier AI developers, (2) registration and reporting requirements to provide regulators with visibility into frontier AI development processes, and (3) mechanisms to ensure compliance with safety standards for the development and deployment of frontier AI models. Industry self-regulation is an important first step. However, wider societal discussions and government intervention will be needed to create standards and to ensure compliance with them. We consider several options to this end, including granting enforcement powers to supervisory authorities and licensure regimes for frontier AI models. Finally, we propose an initial set of safety standards. These include conducting pre-deployment risk assessments; external scrutiny of model behavior; using risk assessments to inform deployment decisions; and monitoring and responding to new information about model capabilities and uses post-deployment. We hope this discussion contributes to the broader conversation on how to balance public safety risks and innovation benefits from advances at the frontier of AI development.

Computers and Society,Artificial Intelligence
It's time for artificial intelligence governance

Suada Hadzovic,Lejla Becirspahic,Sasa Mrdovic

DOI: https://doi.org/10.1016/j.iot.2024.101292

IF: 5.711

2024-10-01

Internet of Things

Abstract:Advances in Internet of Things and Artificial Intelligence have transformed society with the potential to foster the prosperity of human beings and improving societal welfare. However, at the same time, there are concerns about their potential negative impact. In this regard, it is of utmost importance to establish a regulatory framework for the Artificial Intelligence and the Internet of Things that will ensure that human rights and fundamental freedoms are respected, promoted, and protected. In this sense, the goal of this article is to propose the Artificial Intelligence and the Internet of Things Governance model. To achieve a better regulation concept, for the Regulatory Impact Assessment we used the Analytical Hierarchy Process method.
International Institutions for Advanced AI

Lewis Ho,Joslyn Barnhart,Robert Trager,Yoshua Bengio,Miles Brundage,Allison Carnegie,Rumman Chowdhury,Allan Dafoe,Gillian Hadfield,Margaret Levi,Duncan Snidal

2023-07-11

Abstract:International institutions may have an important role to play in ensuring advanced AI systems benefit humanity. International collaborations can unlock AI's ability to further sustainable development, and coordination of regulatory efforts can reduce obstacles to innovation and the spread of benefits. Conversely, the potential dangerous capabilities of powerful and general-purpose AI systems create global externalities in their development and deployment, and international efforts to further responsible AI practices could help manage the risks they pose. This paper identifies a set of governance functions that could be performed at an international level to address these challenges, ranging from supporting access to frontier AI systems to setting international safety standards. It groups these functions into four institutional models that exhibit internal synergies and have precedents in existing organizations: 1) a Commission on Frontier AI that facilitates expert consensus on opportunities and risks from advanced AI, 2) an Advanced AI Governance Organization that sets international standards to manage global threats from advanced models, supports their implementation, and possibly monitors compliance with a future governance regime, 3) a Frontier AI Collaborative that promotes access to cutting-edge AI, and 4) an AI Safety Project that brings together leading researchers and engineers to further AI safety research. We explore the utility of these models and identify open questions about their viability.

Computers and Society
The regulation of artificial intelligence

Finocchiaro, Giusella

DOI: https://doi.org/10.1007/s00146-023-01650-z

2023-04-04

AI & Society

Abstract:Before embarking on a discussion of the regulation of artificial intelligence (AI), it is first necessary to define the subject matter regulated. Defining artificial intelligence is a difficult endeavour, and many definitions have been proposed over the years. Although more than 70 years have passed since it was adopted, the most convincing definition is still nonetheless that proposed by Turing; in any case, it is important to be mindful of the risk of anthropomorphising artificial intelligence, which may arise in particular from its very definition. Once we have established the subject matter regulated, we must ask ourselves whether lawmakers should pursue an approach that seeks to regulate artificial intelligence as a whole, or whether by contrast they should regulate applications of artificial intelligence in specific sectors or individual areas. The proposal for a regulation on artificial intelligence published on 21 April 2021 implements the former approach whilst also pursuing geopolitical goals. After providing an initial overview of the notion of artificial intelligence, this article investigates the geopolitical context to the proposal for a regulation, and then goes on to illustrate the regulatory model embraced by the proposal as well as related critical aspects.
Certified Safe: A Schematic for Approval Regulation of Frontier AI

Cole Salvador

2024-08-12

Abstract:Recent and unremitting capability advances have been accompanied by calls for comprehensive, rather than patchwork, regulation of frontier artificial intelligence (AI). Approval regulation is emerging as a promising candidate. An approval regulation scheme is one in which a firm cannot legally market, or in some cases develop, a product without explicit approval from a regulator on the basis of experiments performed upon the product that demonstrate its safety. This approach is used successfully by the FDA and FAA. Further, its application to frontier AI has been publicly supported by many prominent stakeholders. This report proposes an approval regulation schematic for only the largest AI projects in which scrutiny begins before training and continues through to post-deployment monitoring. The centerpieces of the schematic are two major approval gates, the first requiring approval for large-scale training and the second for deployment. Five main challenges make implementation difficult: noncompliance through unsanctioned deployment, specification of deployment readiness requirements, reliable model experimentation, filtering out safe models before the process, and minimizing regulatory overhead. This report makes a number of crucial recommendations to increase the feasibility of approval regulation, some of which must be followed urgently if such a regime is to succeed in the near future. Further recommendations, produced by this report's analysis, may improve the effectiveness of any regulatory regime for frontier AI.

Computers and Society
Artificial Intelligence Regulation: a framework for governance

Patricia Gomes Rêgo de Almeida,Carlos Denner dos Santos,Josivania Silva Farias

DOI: https://doi.org/10.1007/s10676-021-09593-z

IF: 3.633

2021-04-21

Ethics and Information Technology

Abstract:This article develops a conceptual framework for regulating Artificial Intelligence (AI) that encompasses all stages of modern public policy-making, from the basics to a sustainable governance. Based on a vast systematic review of the literature on Artificial Intelligence Regulation (AIR) published between 2010 and 2020, a dispersed body of knowledge loosely centred around the "framework" concept was organised, described, and pictured for better understanding. The resulting integrative framework encapsulates 21 prior depictions of the policy-making process, aiming to achieve gold-standard societal values, such as fairness, freedom and long-term sustainability. This challenge of integrating the AIR literature was matched by the identification of a structural common ground among different approaches. The AIR framework results from an effort to identify and later analytically deduce synthetic, and generic tool for a country-specific, stakeholder-aware analysis of AIR matters. Theories and principles as diverse as Agile and Ethics were combined in the "AIR framework", which provides a conceptual lens for societies to think collectively and make informed policy decisions related to what, when, and how the uses and applications of AI should be regulated. Moreover, the AIR framework serves as a theoretically sound starting point for endeavours related to AI regulation, from legislation to research and development. As we know, the (potential) impacts of AI on society are immense, and therefore the discourses, social negotiations, and applications of this technology should be guided by common grounds based on contemporary governance techniques, and social values legitimated via dialogue and scientific research.

information science & library science,ethics
Managing extreme AI risks amid rapid progress

Yoshua Bengio,Geoffrey Hinton,Andrew Yao,Dawn Song,Pieter Abbeel,Trevor Darrell,Yuval Noah Harari,Ya-Qin Zhang,Lan Xue,Shai Shalev-Shwartz,Gillian Hadfield,Jeff Clune,Tegan Maharaj,Frank Hutter,Atılım Güneş Baydin,Sheila McIlraith,Qiqi Gao,Ashwin Acharya,David Krueger,Anca Dragan,Philip Torr,Stuart Russell,Daniel Kahneman,Jan Brauner,Sören Mindermann

DOI: https://doi.org/10.1126/science.adn0117

2024-05-23

Abstract:Artificial Intelligence (AI) is progressing rapidly, and companies are shifting their focus to developing generalist AI systems that can autonomously act and pursue goals. Increases in capabilities and autonomy may soon massively amplify AI's impact, with risks that include large-scale social harms, malicious uses, and an irreversible loss of human control over autonomous AI systems. Although researchers have warned of extreme risks from AI, there is a lack of consensus about how exactly such risks arise, and how to manage them. Society's response, despite promising first steps, is incommensurate with the possibility of rapid, transformative progress that is expected by many experts. AI safety research is lagging. Present governance initiatives lack the mechanisms and institutions to prevent misuse and recklessness, and barely address autonomous systems. In this short consensus paper, we describe extreme risks from upcoming, advanced AI systems. Drawing on lessons learned from other safety-critical technologies, we then outline a comprehensive plan combining technical research and development with proactive, adaptive governance mechanisms for a more commensurate preparation.

Computers and Society,Artificial Intelligence,Computation and Language,Machine Learning
Regulating the Safety of Health-Related Artificial Intelligence

Anna Goldenberg,Devin Singh,Michael Silva,Colleen Flood

DOI: https://doi.org/10.12927/hcpol.2022.26824

2022-05-23

Healthcare Policy | Politiques de Santé

Abstract:This article analyzes whether Canada's present approach to regulating health-related artificial intelligence (AI) can address relevant safety-related challenges. Focusing primarily on Health Canada's regulation of medical devices with AI, it examines whether the existing regulatory approach can adequately address general safety concerns, as well as those related to algorithmic bias and challenges posed by the intersections of these concerns with privacy and security interests. It identifies several issues and proposes reforms that aim to ensure that Canadians can access beneficial AI while keeping unsafe products off Canadian markets and motivating safe, effective use of AI products for appropriate purposes and populations.
The Dark Sides of Artificial Intelligence: An Integrated AI Governance Framework for Public Administration

Bernd W. Wirtz,Jan C. Weyerer,Benjamin J. Sturm

DOI: https://doi.org/10.1080/01900692.2020.1749851

2020-04-15

International Journal of Public Administration

Abstract:As government and public administration lag behind the rapid development of AI in their efforts to provide adequate governance, they need respective concepts to keep pace with this dynamic progress. The literature provides few answers to the question of how government and public administration should respond to the great challenges associated with AI and use regulation to prevent harm. This study analyzes AI challenges and former AI regulation approaches. Based on this analysis and regulation theory, an integrated AI governance framework is developed that compiles key aspects of AI governance and provides a guide for the regulatory process of AI and its application. The article concludes with theoretical implications and recommendations for public officers.

English Else
Visibility into AI Agents

Alan Chan,Carson Ezell,Max Kaufmann,Kevin Wei,Lewis Hammond,Herbie Bradley,Emma Bluemke,Nitarshan Rajkumar,David Krueger,Noam Kolt,Lennart Heim,Markus Anderljung

2024-05-18

Abstract:Increased delegation of commercial, scientific, governmental, and personal activities to AI agents -- systems capable of pursuing complex goals with limited supervision -- may exacerbate existing societal risks and introduce new risks. Understanding and mitigating these risks involves critically evaluating existing governance structures, revising and adapting these structures where needed, and ensuring accountability of key stakeholders. Information about where, why, how, and by whom certain AI agents are used, which we refer to as visibility, is critical to these objectives. In this paper, we assess three categories of measures to increase visibility into AI agents: agent identifiers, real-time monitoring, and activity logging. For each, we outline potential implementations that vary in intrusiveness and informativeness. We analyze how the measures apply across a spectrum of centralized through decentralized deployment contexts, accounting for various actors in the supply chain including hardware and software service providers. Finally, we discuss the implications of our measures for privacy and concentration of power. Further work into understanding the measures and mitigating their negative impacts can help to build a foundation for the governance of AI agents.

Computers and Society,Artificial Intelligence
AI and biosecurity: The need for governance

Doni Bloomfield,Jaspreet Pannu,Alex W Zhu,Madelena Y Ng,Ashley Lewis,Eran Bendavid,Steven M Asch,Tina Hernandez-Boussard,Anita Cicero,Tom Inglesby

DOI: https://doi.org/10.1126/science.adq1977

IF: 56.9

2024-08-23

Science

Abstract:Governments should evaluate advanced models and if needed impose safety measures.
Intent-aligned AI systems deplete human agency: the need for agency foundations research in AI safety

Catalin Mitelut,Ben Smith,Peter Vamplew

2023-05-31

Abstract:The rapid advancement of artificial intelligence (AI) systems suggests that artificial general intelligence (AGI) systems may soon arrive. Many researchers are concerned that AIs and AGIs will harm humans via intentional misuse (AI-misuse) or through accidents (AI-accidents). In respect of AI-accidents, there is an increasing effort focused on developing algorithms and paradigms that ensure AI systems are aligned to what humans intend, e.g. AI systems that yield actions or recommendations that humans might judge as consistent with their intentions and goals. Here we argue that alignment to human intent is insufficient for safe AI systems and that preservation of long-term agency of humans may be a more robust standard, and one that needs to be separated explicitly and a priori during optimization. We argue that AI systems can reshape human intention and discuss the lack of biological and psychological mechanisms that protect humans from loss of agency. We provide the first formal definition of agency-preserving AI-human interactions which focuses on forward-looking agency evaluations and argue that AI systems - not humans - must be increasingly tasked with making these evaluations. We show how agency loss can occur in simple environments containing embedded agents that use temporal-difference learning to make action recommendations. Finally, we propose a new area of research called "agency foundations" and pose four initial topics designed to improve our understanding of agency in AI-human interactions: benevolent game theory, algorithmic foundations of human rights, mechanistic interpretability of agency representation in neural-networks and reinforcement learning from internal states.

Artificial Intelligence,Computers and Society,Human-Computer Interaction
Regulatory Markets: The Future of AI Governance

Gillian K. Hadfield,Jack Clark

2023-04-26

Abstract:Appropriately regulating artificial intelligence is an increasingly urgent policy challenge. Legislatures and regulators lack the specialized knowledge required to best translate public demands into legal requirements. Overreliance on industry self-regulation fails to hold producers and users of AI systems accountable to democratic demands. Regulatory markets, in which governments require the targets of regulation to purchase regulatory services from a private regulator, are proposed. This approach to AI regulation could overcome the limitations of both command-and-control regulation and self-regulation. Regulatory market could enable governments to establish policy priorities for the regulation of AI, whilst relying on market forces and industry R&D efforts to pioneer the methods of regulation that best achieve policymakers' stated objectives.

Artificial Intelligence,General Economics
Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems

David "davidad" Dalrymple,Joar Skalse,Yoshua Bengio,Stuart Russell,Max Tegmark,Sanjit Seshia,Steve Omohundro,Christian Szegedy,Ben Goldhaber,Nora Ammann,Alessandro Abate,Joe Halpern,Clark Barrett,Ding Zhao,Tan Zhi-Xuan,Jeannette Wing,Joshua Tenenbaum

2024-07-08

Abstract:Ensuring that AI systems reliably and robustly avoid harmful or dangerous behaviours is a crucial challenge, especially for AI systems with a high degree of autonomy and general intelligence, or systems used in safety-critical contexts. In this paper, we will introduce and define a family of approaches to AI safety, which we will refer to as guaranteed safe (GS) AI. The core feature of these approaches is that they aim to produce AI systems which are equipped with high-assurance quantitative safety guarantees. This is achieved by the interplay of three core components: a world model (which provides a mathematical description of how the AI system affects the outside world), a safety specification (which is a mathematical description of what effects are acceptable), and a verifier (which provides an auditable proof certificate that the AI satisfies the safety specification relative to the world model). We outline a number of approaches for creating each of these three core components, describe the main technical challenges, and suggest a number of potential solutions to them. We also argue for the necessity of this approach to AI safety, and for the inadequacy of the main alternative approaches.

Artificial Intelligence
Could regulating the creators deliver trustworthy AI?

Labhaoise Ni Fhaolain,Andrew Hines

DOI: https://doi.org/10.48550/arXiv.2006.14750

2020-06-26

Abstract:Is a new regulated profession, such as Artificial Intelligence (AI) Architect who is responsible and accountable for AI outputs necessary to ensure trustworthy AI? AI is becoming all pervasive and is often deployed in everyday technologies, devices and services without our knowledge. There is heightened awareness of AI in recent years which has brought with it fear. This fear is compounded by the inability to point to a trustworthy source of AI, however even the term "trustworthy AI" itself is troublesome. Some consider trustworthy AI to be that which complies with relevant laws, while others point to the requirement to comply with ethics and standards (whether in addition to or in isolation of the law). This immediately raises questions of whose ethics and which standards should be applied and whether these are sufficient to produce trustworthy AI in any event.

Computers and Society
Analyzing Potential Solutions Involving Regulation to Escape Some of AI's Ethical Concerns

Jay Nemec

2024-03-22

Abstract:Artificial intelligence (AI), although not able to currently capture the many complexities of humans, are slowly adapting to have certain capabilities of humans, many of which can revolutionize our world. AI systems, such as ChatGPT and others utilized within various industries for specific processes, have been transforming rapidly. However, this transformation can occur in an extremely concerning way if certain measures are not taken. This article touches on some of the current issues within the artificial intelligence ethical crisis, such as the concerns of discrimination within AI and false information that is becoming readily available with AI. Within this article, plausible solutions involving regulation are discussed and how they would mitigate ethical concerns. These include the self-regulation of businesses along with government regulation, and the effects these possible solutions can both have on current AI concerns.

Computers and Society

Regulating advanced artificial agents

Appraising Regulatory Framework Towards Artificial General Intelligence (AGI) under Digital Humanism

Artificially Intelligent Regulation

Regulating artificial intelligence: Proposal for a global solution

Frontier AI Regulation: Managing Emerging Risks to Public Safety

It's time for artificial intelligence governance

International Institutions for Advanced AI

The regulation of artificial intelligence

Certified Safe: A Schematic for Approval Regulation of Frontier AI

Artificial Intelligence Regulation: a framework for governance

Managing extreme AI risks amid rapid progress

Regulating the Safety of Health-Related Artificial Intelligence

The Dark Sides of Artificial Intelligence: An Integrated AI Governance Framework for Public Administration

Visibility into AI Agents

AI and biosecurity: The need for governance

Intent-aligned AI systems deplete human agency: the need for agency foundations research in AI safety

Regulatory Markets: The Future of AI Governance

Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems

Could regulating the creators deliver trustworthy AI?

Analyzing Potential Solutions Involving Regulation to Escape Some of AI's Ethical Concerns