AI and biosecurity: The need for governance

Doni Bloomfield,Jaspreet Pannu,Alex W Zhu,Madelena Y Ng,Ashley Lewis,Eran Bendavid,Steven M Asch,Tina Hernandez-Boussard,Anita Cicero,Tom Inglesby

DOI: https://doi.org/10.1126/science.adq1977

IF: 56.9

2024-08-23

Science

Abstract:Governments should evaluate advanced models and if needed impose safety measures.

What problem does this paper attempt to address?

Prioritizing High-Consequence Biological Capabilities in Evaluations of Artificial Intelligence Models

Jaspreet Pannu,Doni Bloomfield,Alex Zhu,Robert MacKnight,Gabe Gomes,Anita Cicero,Thomas V. Inglesby

DOI: https://doi.org/10.2139/ssrn.4873106

2024-07-23

Abstract:As a result of rapidly accelerating AI capabilities, over the past year, national governments and multinational bodies have announced efforts to address safety, security and ethics issues related to AI models. One high priority among these efforts is the mitigation of misuse of AI models. Many biologists have for decades sought to reduce the risks of scientific research that could lead, through accident or misuse, to high-consequence disease outbreaks. Scientists have carefully considered what types of life sciences research have the potential for both benefit and risk (dual-use), especially as scientific advances have accelerated our ability to engineer organisms and create novel variants of pathogens. Here we describe how previous experience and study by scientists and policy professionals of dual-use capabilities in the life sciences can inform risk evaluations of AI models with biological capabilities. We argue that AI model evaluations should prioritize addressing high-consequence risks (those that could cause large-scale harm to the public, such as pandemics), and that these risks should be evaluated prior to model deployment so as to allow potential biosafety and/or biosecurity measures. Scientists' experience with identifying and mitigating dual-use biological risks can help inform new approaches to evaluating biological AI models. Identifying which AI capabilities post the greatest biosecurity and biosafety concerns is necessary in order to establish targeted AI safety evaluation methods, secure these tools against accident and misuse, and avoid impeding immense potential benefits.

Computers and Society,Artificial Intelligence,Emerging Technologies
Artificial intelligence challenges in the face of biological threats: emerging catastrophic risks for public health

Renan Chaves de Lima,Lucas Sinclair,Ricardo Megger,Magno Alessandro Guedes Maciel,Pedro Fernando da Costa Vasconcelos,Juarez Antônio Simões Quaresma

DOI: https://doi.org/10.3389/frai.2024.1382356

2024-05-10

Abstract:The threat landscape of biological hazards with the evolution of AI presents challenges. While AI promises innovative solutions, concerns arise about its misuse in the creation of biological weapons. The convergence of AI and genetic editing raises questions about biosecurity, potentially accelerating the development of dangerous pathogens. The mapping conducted highlights the critical intersection between AI and biological threats, underscoring emerging risks in the criminal manipulation of pathogens. Technological advancement in biology requires preventative and regulatory measures. Expert recommendations emphasize the need for solid regulations and responsibility of creators, demanding a proactive, ethical approach and governance to ensure global safety.
It's time for artificial intelligence governance

Suada Hadzovic,Lejla Becirspahic,Sasa Mrdovic

DOI: https://doi.org/10.1016/j.iot.2024.101292

IF: 5.711

2024-10-01

Internet of Things

Abstract:Advances in Internet of Things and Artificial Intelligence have transformed society with the potential to foster the prosperity of human beings and improving societal welfare. However, at the same time, there are concerns about their potential negative impact. In this regard, it is of utmost importance to establish a regulatory framework for the Artificial Intelligence and the Internet of Things that will ensure that human rights and fundamental freedoms are respected, promoted, and protected. In this sense, the goal of this article is to propose the Artificial Intelligence and the Internet of Things Governance model. To achieve a better regulation concept, for the Regulatory Impact Assessment we used the Analytical Hierarchy Process method.
Biosecurity Risk Assessment for the Use of Artificial Intelligence in Synthetic Biology

Leyma P De Haro

DOI: https://doi.org/10.1089/apb.2023.0031

2024-06-20

Abstract:Background: The integration of Artificial Intelligence (AI) with synthetic biology is driving unprecedented progress in both fields. However, this integration introduces complex biosecurity challenges. Addressing these concerns, this article proposes a specialized biosecurity risk assessment process designed to evaluate the incorporation of AI in synthetic biology. Methods: A set of tailored tools and methodology was developed for conducting biosecurity risk assessments of AI language models used for synthetic biology. These resources were developed to guide risk management professionals through a systematic process of identifying, evaluating, and mitigating potential risks. Results: The tools and methodology provided offer a structured approach to risk assessment, enabling risk management professionals to comprehensively analyze the biosecurity implications of AI applications in synthetic biology. They facilitate the identification of potential risks and the development of effective mitigation strategies. An example of a risk assessment performed on the large language model "ChatGPT 4.0" is provided here. Conclusion: AI's role in synthetic biology is rapidly expanding; thus, establishing proactive and secure practices is crucial. The biosecurity risk assessment tools and methodology presented here are the first provided in the literature and will be instrumental steps toward the responsible integration of AI in synthetic biology. By adopting these resources, the biorisk management community can effectively navigate and manage the biosecurity challenges posed by AI, ensuring its responsible and secure application in the field of synthetic biology.
Modern International Issues and Prospective Solutions in the Field Of Genomics and AI Regulation: Creating a System for Assessing Safety and Ethics of AI Technologies

Ekaterina Kan

DOI: https://doi.org/10.59022/ujldp.117

2023-09-14

Abstract:With the rapid development of genomics and artificial intelligence (AI) technologies worldwide, there is an urgent need for strengthened international regulation and cooperation to ensure these innovations are developed and deployed safely, ethically, and for the benefit of humanity. This article provides an overview of key gaps and risks in the current governance of AI and genomics across national jurisdictions, such as biased decision-making encoded in algorithms, lack of accountability mechanisms, and potential for human rights violations. It proposes the establishment of an international organization mandated to develop standardized testing protocols and evidence-based guidelines for assessing the safety, security, and ethics of AI systems. Such an organization would convene diverse representatives from governments, industry, academia, and civil society to build consensus on responsible governance approaches aligned with shared human values and international law. This article analyzes pressing issues arising from fragmented regulatory regimes and offers solutions to advance multilateral cooperation, oversight, and public trust in emerging technologies.
The whack-a-mole governance challenge for AI-enabled synthetic biology: literature review and emerging frameworks

Trond Arne Undheim

DOI: https://doi.org/10.3389/fbioe.2024.1359768

2024-02-01

Abstract:AI-enabled synthetic biology has tremendous potential but also significantly increases biorisks and brings about a new set of dual use concerns. The picture is complicated given the vast innovations envisioned to emerge by combining emerging technologies, as AI-enabled synthetic biology potentially scales up bioengineering into industrial biomanufacturing. However, the literature review indicates that goals such as maintaining a reasonable scope for innovation, or more ambitiously to foster a huge bioeconomy don't necessarily contrast with biosafety, but need to go hand in hand. This paper presents a literature review of the issues and describes emerging frameworks for policy and practice that transverse the options of command-and control, stewardship, bottom-up, and laissez-faire governance. How to achieve early warning systems that enable prevention and mitigation of future AI-enabled biohazards from the lab, from deliberate misuse, or from the public realm, will constantly need to evolve, and adaptive, interactive approaches should emerge. Although biorisk is subject to an established governance regime, and scientists generally adhere to biosafety protocols, even experimental, but legitimate use by scientists could lead to unexpected developments. Recent advances in chatbots enabled by generative AI have revived fears that advanced biological insight can more easily get into the hands of malignant individuals or organizations. Given these sets of issues, society needs to rethink how AI-enabled synthetic biology should be governed. The suggested way to visualize the challenge at hand is whack-a-mole governance, although the emerging solutions are perhaps not so different either.

Other Quantitative Biology,Artificial Intelligence
AI Governance: A Challenge for Public Health

Jennifer K Wagner,Megan Doerr,Cason D Schmit

DOI: https://doi.org/10.2196/58358

2024-10-01

JMIR Public Health and Surveillance

Abstract:Rapidly evolving artificial intelligence (AI) is capable of structuralizing social, political, and economic determinants of health into the invisible algorithms that shape every facet of modern life. Nevertheless, AI could be a powerful public health tool, enabling beneficial objectives like precision public health and medicine. Developing an AI governance framework that can maximize the benefits and minimize the risks of AI is a significant challenge. The benefits of public health engagement in AI governance could be extensive. Here, we describe how several public health concepts could enhance AI governance. Specifically, we explain how (1) harm reduction provides framework for navigating the governance debate between traditional regulation and "soft law" approaches; (2) a public health understanding of social determinants of health is essential to optimally weigh the potential risks and benefits of AI; (3) public health ethics provides a toolset for guiding governance decisions where individual interests intersect with collective interests; and (5) a One Health approach to AI governance can both improve governance effectiveness while advancing public health outcomes. Public health theories, perspectives, and innovations could substantially enrich and improve AI governance, creating a more equitable and socially beneficial path for AI development.

public, environmental & occupational health
Challenges and recent progress in the governance of biosecurity risks in the era of synthetic biology

Tao Sun,Jie Song,Meng Wang,Chao Zhao,Weiwen Zhang

DOI: https://doi.org/10.1016/j.jobb.2022.02.002

2022-06-01

Journal of Biosafety and Biosecurity

Abstract:Considerable progress has been achieved in basic research and the biotechnological application of biological sciences in recent years. Synthetic biology integrates systems biology, engineering, computer science, and other disciplines to achieve the “modification of life” or even the “creation of life” via the redesign of existing natural systems or the development of new biological components and devices. However, the research and application of synthetic biology can create potential risks, such as aggravation of species with complex gene modifications, threats to species diversity, abuse of biological weapons, laboratory leaks, and man-made mutations. Without a suitable governance system, such research activities could result in harm to humans, plants, and animals, as well as to natural ecological systems. In this article, we first briefly summarize technical progress in synthetic biology in recent years and the potential bioethical and biosecurity risks, and then describe current international treaties, guidance documents, and national regulatory measures designed to address potential harm caused by the dual-use property of synthetic biology, including the Biological Weapons Convention, the Convention on Biological Diversity, and the Model Code of Conduct for Biological Scientists (Tianjin Biosecurity Guidelines for Codes of Conduct for Scientists). In addition, we also present some recommendations for better governance of synthetic biology research and applications in China, including strengthening the biosecurity capacity, improving the biosecurity regulatory system, and promoting multilevel international cooperation to effectively address the potential biosecurity risks of synthetic biology.

English Else
Governing dual-use technologies: Case studies of international security agreements and lessons for AI governance

Akash R. Wasil,Peter Barnett,Michael Gerovitch,Roman Hauksson,Tom Reed,Jack William Miller

2024-09-04

Abstract:International AI governance agreements and institutions may play an important role in reducing global security risks from advanced AI. To inform the design of such agreements and institutions, we conducted case studies of historical and contemporary international security agreements. We focused specifically on those arrangements around dual-use technologies, examining agreements in nuclear security, chemical weapons, biosecurity, and export controls. For each agreement, we examined four key areas: (a) purpose, (b) core powers, (c) governance structure, and (d) instances of non-compliance. From these case studies, we extracted lessons for the design of international AI agreements and governance institutions. We discuss the importance of robust verification methods, strategies for balancing power between nations, mechanisms for adapting to rapid technological change, approaches to managing trade-offs between transparency and security, incentives for participation, and effective enforcement mechanisms.

Computers and Society,Artificial Intelligence
Regulating advanced artificial agents

Michael K Cohen,Noam Kolt,Yoshua Bengio,Gillian K Hadfield,Stuart Russell,Michael K. Cohen,Gillian K. Hadfield

DOI: https://doi.org/10.1126/science.adl0625

IF: 56.9

2024-04-05

Science

Abstract:Governance frameworks should address the prospect of AI systems that cannot be safely tested

multidisciplinary sciences
Artificial intelligence and biological misuse: Differentiating risks of language models and biological design tools

Jonas B. Sandbrink

2023-12-23

Abstract:As advancements in artificial intelligence (AI) propel progress in the life sciences, they may also enable the weaponisation and misuse of biological agents. This article differentiates two classes of AI tools that could pose such biosecurity risks: large language models (LLMs) and biological design tools (BDTs). LLMs, such as GPT-4 and its successors, might provide dual-use information and thus remove some barriers encountered by historical biological weapons efforts. As LLMs are turned into multi-modal lab assistants and autonomous science tools, this will increase their ability to support non-experts in performing laboratory work. Thus, LLMs may in particular lower barriers to biological misuse. In contrast, BDTs will expand the capabilities of sophisticated actors. Concretely, BDTs may enable the creation of pandemic pathogens substantially worse than anything seen to date and could enable forms of more predictable and targeted biological weapons. In combination, the convergence of LLMs and BDTs could raise the ceiling of harm from biological agents and could make them broadly accessible. A range of interventions would help to manage risks. Independent pre-release evaluations could help understand the capabilities of models and the effectiveness of safeguards. Options for differentiated access to such tools should be carefully weighed with the benefits of openly releasing systems. Lastly, essential for mitigating risks will be universal and enhanced screening of gene synthesis products.

Computers and Society
Towards Data Governance of Frontier AI Models

Jason Hausenloy,Duncan McClements,Madhavendra Thakur

2024-12-05

Abstract:Data is essential to train and fine-tune today's frontier artificial intelligence (AI) models and to develop future ones. To date, academic, legal, and regulatory work has primarily addressed how data can directly harm consumers and creators, such as through privacy breaches, copyright infringements, and bias and discrimination. Our work, instead, focuses on the comparatively neglected question of how data can enable new governance capacities for frontier AI models. This approach for "frontier data governance" opens up new avenues for monitoring and mitigating risks from advanced AI models, particularly as they scale and acquire specific dangerous capabilities. Still, frontier data governance faces challenges that stem from the fundamental properties of data itself: data is non-rival, often non-excludable, easily replicable, and increasingly synthesizable. Despite these inherent difficulties, we propose a set of policy mechanisms targeting key actors along the data supply chain, including data producers, aggregators, model developers, and data vendors. We provide a brief overview of 15 governance mechanisms, of which we centrally introduce five, underexplored policy recommendations. These include developing canary tokens to detect unauthorized use for producers; (automated) data filtering to remove malicious content for pre-training and post-training datasets; mandatory dataset reporting requirements for developers and vendors; improved security for datasets and data generation algorithms; and know-your-customer requirements for vendors. By considering data not just as a source of potential harm, but as a critical governance lever, this work aims to equip policymakers with a new tool for the governance and regulation of frontier AI models.

Artificial Intelligence
Towards Risk Analysis of the Impact of AI on the Deliberate Biological Threat Landscape

Matthew E. Walsh

2024-06-11

Abstract:The perception that the convergence of biological engineering and artificial intelligence (AI) could enable increased biorisk has recently drawn attention to the governance of biotechnology and artificial intelligence. The 2023 Executive Order, Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence, requires an assessment of how artificial intelligence can increase biorisk. Within this perspective, quantitative and qualitative frameworks for evaluating biorisk are presented. Both frameworks are exercised using notional scenarios and their benefits and limitations are then discussed. Finally, the perspective concludes by noting that assessment and evaluation methodologies must keep pace with advances of AI in the life sciences.

Computers and Society
The Convergence of AI and Synthetic Biology: The Looming Deluge

Cindy Vindman,Benjamin Trump,Christopher Cummings,Madison Smith,Alexander J. Titus,Ken Oye,Valentina Prado,Eyup Turmus,Igor Linkov

2024-04-29

Abstract:The convergence of artificial intelligence (AI) and synthetic biology is rapidly accelerating the pace of biological discovery and engineering. AI techniques, such as large language models and biological design tools, are enabling the automated design, build, test, and learning cycles for engineered biological systems. This convergence promises to democratize synthetic biology and unlock novel applications across domains from medicine to environmental sustainability. However, it also poses significant risks around reliability, dual use, and governance. The opacity of AI models, the deskilling of workforces, and the outdated nature of current regulatory frameworks present challenges in ensuring responsible development. Urgent attention is needed to update governance structures, integrate human oversight into increasingly automated workflows, and foster a culture of responsibility among the growing community of bioengineers. Only by proactively addressing these issues can we realize the transformative potential of AI-driven synthetic biology while mitigating its risks.

Other Quantitative Biology
The Reality of AI and Biorisk

Aidan Peppin,Anka Reuel,Stephen Casper,Elliot Jones,Andrew Strait,Usman Anwar,Anurag Agrawal,Sayash Kapoor,Sanmi Koyejo,Marie Pellat,Rishi Bommasani,Nick Frosst,Sara Hooker

2024-12-04

Abstract:To accurately and confidently answer the question 'could an AI model or system increase biorisk', it is necessary to have both a sound theoretical threat model for how AI models or systems could increase biorisk and a robust method for testing that threat model. This paper provides an analysis of existing available research surrounding two AI and biorisk threat models: 1) access to information and planning via large language models (LLMs), and 2) the use of AI-enabled biological tools (BTs) in synthesizing novel biological artifacts. We find that existing studies around AI-related biorisk are nascent, often speculative in nature, or limited in terms of their methodological maturity and transparency. The available literature suggests that current LLMs and BTs do not pose an immediate risk, and more work is needed to develop rigorous approaches to understanding how future models could increase biorisks. We end with recommendations about how empirical work can be expanded to more precisely target biorisk and ensure rigor and validity of findings.

Artificial Intelligence
Regulating artificial intelligence: Proposal for a global solution

Olivia J. Erdélyi,Judy Goldsmith

DOI: https://doi.org/10.1016/j.giq.2022.101748

IF: 8.49

2022-10-01

Government Information Quarterly

Abstract:With increasing ubiquity of artificial intelligence (AI) in modern societies, individual countries and the international community are working hard to create an innovation-friendly, yet safe, regulatory environment. Adequate regulation is key to maximize the benefits and minimize the risks stemming from AI technologies. Developing regulatory frameworks is, however, challenging due to AI's global reach, agency problems present in regulation, and the existence of widespread misconceptions about the very notion of regulation. This paper makes three claims: (1) Based on interdisciplinary insights, we show that AI-related challenges cannot be tackled effectively without sincere international coordination supported by robust, consistent domestic, regional, and international governance arrangements. (2) Against this backdrop, we propose the establishment of an international AI governance framework to spearhead initiatives to create a consistent, global enabling regulatory environment, which is necessary for the successful and responsible adoption of AI technologies. To facilitate the practical implementation of our recommendation, we provide a simplified impact assessment on regulatory architecture and governance design options, appropriate to the scope of the paper. (3) We draw attention to communication challenges, which we believe are underestimated barriers hindering contemporary efforts to develop AI regulatory regimes. We argue that a fundamental change of mindset regarding the nature of regulation is necessary to remove these, and put forward some recommendations on how to achieve this.

information science & library science
Data-Centric AI Governance: Addressing the Limitations of Model-Focused Policies

Ritwik Gupta,Leah Walker,Rodolfo Corona,Stephanie Fu,Suzanne Petryk,Janet Napolitano,Trevor Darrell,Andrew W. Reddie

2024-09-26

Abstract:Current regulations on powerful AI capabilities are narrowly focused on "foundation" or "frontier" models. However, these terms are vague and inconsistently defined, leading to an unstable foundation for governance efforts. Critically, policy debates often fail to consider the data used with these models, despite the clear link between data and model performance. Even (relatively) "small" models that fall outside the typical definitions of foundation and frontier models can achieve equivalent outcomes when exposed to sufficiently specific datasets. In this work, we illustrate the importance of considering dataset size and content as essential factors in assessing the risks posed by models both today and in the future. More broadly, we emphasize the risk posed by over-regulating reactively and provide a path towards careful, quantitative evaluation of capabilities that can lead to a simplified regulatory environment.

Computers and Society,Artificial Intelligence
Frontier AI Regulation: Managing Emerging Risks to Public Safety

Markus Anderljung,Joslyn Barnhart,Anton Korinek,Jade Leung,Cullen O'Keefe,Jess Whittlestone,Shahar Avin,Miles Brundage,Justin Bullock,Duncan Cass-Beggs,Ben Chang,Tantum Collins,Tim Fist,Gillian Hadfield,Alan Hayes,Lewis Ho,Sara Hooker,Eric Horvitz,Noam Kolt,Jonas Schuett,Yonadav Shavit,Divya Siddarth,Robert Trager,Kevin Wolf

2023-11-08

Abstract:Advanced AI models hold the promise of tremendous benefits for humanity, but society needs to proactively manage the accompanying risks. In this paper, we focus on what we term "frontier AI" models: highly capable foundation models that could possess dangerous capabilities sufficient to pose severe risks to public safety. Frontier AI models pose a distinct regulatory challenge: dangerous capabilities can arise unexpectedly; it is difficult to robustly prevent a deployed model from being misused; and, it is difficult to stop a model's capabilities from proliferating broadly. To address these challenges, at least three building blocks for the regulation of frontier models are needed: (1) standard-setting processes to identify appropriate requirements for frontier AI developers, (2) registration and reporting requirements to provide regulators with visibility into frontier AI development processes, and (3) mechanisms to ensure compliance with safety standards for the development and deployment of frontier AI models. Industry self-regulation is an important first step. However, wider societal discussions and government intervention will be needed to create standards and to ensure compliance with them. We consider several options to this end, including granting enforcement powers to supervisory authorities and licensure regimes for frontier AI models. Finally, we propose an initial set of safety standards. These include conducting pre-deployment risk assessments; external scrutiny of model behavior; using risk assessments to inform deployment decisions; and monitoring and responding to new information about model capabilities and uses post-deployment. We hope this discussion contributes to the broader conversation on how to balance public safety risks and innovation benefits from advances at the frontier of AI development.

Computers and Society,Artificial Intelligence
Towards Responsible Governance of Biological Design Tools

Richard Moulange,Max Langenkamp,Tessa Alexanian,Samuel Curtis,Morgan Livingston

DOI: https://doi.org/10.48550/arXiv.2311.15936

2023-11-27

Computers and Society

Abstract:Recent advancements in generative machine learning have enabled rapid progress in biological design tools (BDTs) such as protein structure and sequence prediction models. The unprecedented predictive accuracy and novel design capabilities of BDTs present new and significant dual-use risks. For example, their predictive accuracy allows biological agents, whether vaccines or pathogens, to be developed more quickly, while the design capabilities could be used to discover drugs or evade DNA screening techniques. Similar to other dual-use AI systems, BDTs present a wicked problem: how can regulators uphold public safety without stifling innovation? We highlight how current regulatory proposals that are primarily tailored toward large language models may be less effective for BDTs, which require fewer computational resources to train and are often developed in an open-source manner. We propose a range of measures to mitigate the risk that BDTs are misused, across the areas of responsible development, risk assessment, transparency, access management, cybersecurity, and investing in resilience. Implementing such measures will require close coordination between developers and governments.

AI and biosecurity: The need for governance

Prioritizing High-Consequence Biological Capabilities in Evaluations of Artificial Intelligence Models

Artificial intelligence challenges in the face of biological threats: emerging catastrophic risks for public health

It's time for artificial intelligence governance

Biosecurity Risk Assessment for the Use of Artificial Intelligence in Synthetic Biology

Modern International Issues and Prospective Solutions in the Field Of Genomics and AI Regulation: Creating a System for Assessing Safety and Ethics of AI Technologies

The whack-a-mole governance challenge for AI-enabled synthetic biology: literature review and emerging frameworks

AI Governance: A Challenge for Public Health

Challenges and recent progress in the governance of biosecurity risks in the era of synthetic biology

Governing dual-use technologies: Case studies of international security agreements and lessons for AI governance

Regulating advanced artificial agents

Artificial intelligence and biological misuse: Differentiating risks of language models and biological design tools

Towards Data Governance of Frontier AI Models

Towards Risk Analysis of the Impact of AI on the Deliberate Biological Threat Landscape

The Convergence of AI and Synthetic Biology: The Looming Deluge

The Reality of AI and Biorisk

Regulating artificial intelligence: Proposal for a global solution

Data-Centric AI Governance: Addressing the Limitations of Model-Focused Policies

Frontier AI Regulation: Managing Emerging Risks to Public Safety

Towards Responsible Governance of Biological Design Tools