Security challenges by AI-assisted protein design

Philip Hunter

DOI: https://doi.org/10.1038/s44319-024-00124-7

IF: 9.071

2024-03-26

EMBO Reports

Abstract:Scientists and security experts are concerned that the increasing power of AI-assisted protein design and synthesis could be abused by various actors for terrorist or criminal purposes.

cell biology,biochemistry & molecular biology

What problem does this paper attempt to address?

Protein design meets biosecurity

David Baker,George Church

DOI: https://doi.org/10.1126/science.ado1671

IF: 56.9

2024-01-26

Science

Abstract:The power and accuracy of computational protein design have been increasing rapidly with the incorporation of artificial intelligence (AI) approaches. This promises to transform biotechnology, enabling advances across sustainability and medicine. DNA synthesis plays a critical role in materializing designed proteins. However, as with all major revolutionary changes, this technology is vulnerable to misuse and the production of dangerous biological agents. To enable the full benefits of this revolution while mitigating risks that may emerge, all synthetic gene sequence and synthesis data should be collected and stored in repositories that are only queried in emergencies to ensure that protein design proceeds in a safe, secure, and trustworthy manner.

multidisciplinary sciences
Could AI-designed proteins be weaponized? Scientists lay out safety guidelines

Ewen Callaway

DOI: https://doi.org/10.1038/d41586-024-00699-0

IF: 64.8

2024-03-09

Nature

Abstract:AI tools that can come up with protein structures at the push of a button should be used safely and ethically, say researchers in the field. AI tools that can come up with protein structures at the push of a button should be used safely and ethically, say researchers in the field.

multidisciplinary sciences
Toward AI-Resilient Screening of Nucleic Acid Synthesis Orders: Process, Results, and Recommendations

Bruce J. Wittmann,Tessa Alexanian,Craig Bartling,Jacob Beal,Adam Clore,James Diggans,Kevin Flyangolts,Bryan T. Gemler,Tom Mitchell,Steven T. Murphy,Nicole E. Wheeler,Eric Horvitz

DOI: https://doi.org/10.1101/2024.12.02.626439

2024-12-04

Abstract:Fast-moving advances in AI-assisted protein engineering are enabling breakthroughs in the life sciences that promise numerous beneficial applications. At the same time, these new capabilities are creating potential biosecurity challenges by providing new pathways to intentional or accidental synthesis of genes that encode hazardous proteins. The synthesis of nucleic acids is a key choke point in the AI-assisted protein engineering pipeline as it is where digital designs are transformed into physical instructions that can produce potentially harmful proteins. Thus, one focus for efforts to enhance biosecurity in the face of new AI-enabled capabilities is on bolstering the screening of orders by nucleic acid synthesis providers. We describe a multistakeholder, cross-sector effort to address biosecurity challenges with uses of AI-powered biological design tools to reformulate naturally occurring proteins of concern to create synthetic homologs that have low sequence identity to the wild-type proteins. We evaluated the abilities of traditional nucleic acid biosecurity screening tools to detect these synthetic homologs and found that, of tools tested, not all could previously detect such AI-redesigned sequences reliably. However, as we report, patches were built and deployed to improve detection rates over the course of the project, resulting in a final mean detection rate over tools of 97% of the synthetic homologs that were determined, using in-silico metrics, to be more likely to retain wild-type-like function. Finally, we make recommendations on approaches for studying and addressing the rising risk of adversarial AI-assisted protein engineering attacks like the one we identified and worked to mitigate.

Synthetic Biology
Threats and Opportunities with AI-based Cyber Security Intrusion Detection: A Review

Bibhu Dash,Meraj Farheen Ansari,Pawankumar Sharma,Azad Ali

DOI: https://doi.org/10.5121/ijsea.2022.13502

2022-09-30

International Journal of Software Engineering & Applications

Abstract:Internet usage has increased quickly, particularly in the previous decade. With the widespread use of the internet, cybercrime is growing at an alarming rate in our daily lives. However, with the growth of artificial intelligence (AI), businesses are concentrating more on preventing cybercrime. AI is becoming an essential component of every business, affecting individuals worldwide. Cybercrime is one of the most prominent domains where AI has begun demonstrating valuable inputs. As a result, AI is being deployed as the first line of defense in most firms' systems. Because AI can detect new assaults faster than humans, it is the best alternative for constructing better protection against cybercrime. AI technologies also offer more significant potential in the development of such technology. This paper discusses recent cyber intrusions and how the AI-enabled industry is preparing to defend itself in the long run.
Dual use of artificial-intelligence-powered drug discovery

Fabio Urbina,Filippa Lentzos,Cédric Invernizzi,Sean Ekins

DOI: https://doi.org/10.1038/s42256-022-00465-9

IF: 23.8

2022-03-01

Nature Machine Intelligence

Abstract:Nature Machine Intelligence, Published online: 07 March 2022; doi:10.1038/s42256-022-00465-9An international security conference explored how artificial intelligence (AI) technologies for drug discovery could be misused for de novo design of biochemical weapons. A thought experiment evolved into a computational proof.

computer science, artificial intelligence, interdisciplinary applications
Peptide Toxins as Biothreats and the Potential for AI Systems to Enhance Biosecurity

Ying-Chiang J Lee,Alexis Cowan,Amari Tankard

DOI: https://doi.org/10.3389/fbioe.2022.860390

2022-03-08

Abstract:Biological weapons have been used for thousands of years, but recent advances in synthesis technologies have made peptide and protein toxin production more accessible and pose a threat to biosecurity worldwide. Natural toxins such as conotoxins, certain hemolytic compounds, and enterotoxins are peptide agents that can be synthesized in an environment with weak biosecurity measures and rudimentarily weaponized for limited use against smaller targets for lethal or nonlethal effects. Technological advances are changing the threat landscape around biological weapons and potentially facilitating a shift from state sponsored to more micro-level threats stemming from terror cells, insider threats, and lone wolf attacks. Here, we present the reader with an overview of the threat of peptide and protein toxins, provide examples of potent peptide toxins, and introduce capabilities of a proposed biosecurity program utilizing artificial intelligence that unifies commercial nucleotide and peptide synthesis vendors.
Artificial intelligence challenges in the face of biological threats: emerging catastrophic risks for public health

Renan Chaves de Lima,Lucas Sinclair,Ricardo Megger,Magno Alessandro Guedes Maciel,Pedro Fernando da Costa Vasconcelos,Juarez Antônio Simões Quaresma

DOI: https://doi.org/10.3389/frai.2024.1382356

2024-05-10

Abstract:The threat landscape of biological hazards with the evolution of AI presents challenges. While AI promises innovative solutions, concerns arise about its misuse in the creation of biological weapons. The convergence of AI and genetic editing raises questions about biosecurity, potentially accelerating the development of dangerous pathogens. The mapping conducted highlights the critical intersection between AI and biological threats, underscoring emerging risks in the criminal manipulation of pathogens. Technological advancement in biology requires preventative and regulatory measures. Expert recommendations emphasize the need for solid regulations and responsibility of creators, demanding a proactive, ethical approach and governance to ensure global safety.
Artificial Intelligence Security: Threats and Countermeasures

Yupeng Hu,Wenxin Kuang,Zheng Qin,Kenli Li,Jiliang Zhang,Yansong Gao,Wenjia Li,Keqin Li

DOI: https://doi.org/10.1145/3487890

IF: 16.6

2023-01-31

ACM Computing Surveys

Abstract:In recent years, with rapid technological advancement in both computing hardware and algorithm, Artificial Intelligence (AI) has demonstrated significant advantage over human being in a wide range of fields, such as image recognition, education, autonomous vehicles, finance, and medical diagnosis. However, AI-based systems are generally vulnerable to various security threats throughout the whole process, ranging from the initial data collection and preparation to the training, inference, and final deployment. In an AI-based system, the data collection and pre-processing phase are vulnerable to sensor spoofing attacks and scaling attacks, respectively, while the training and inference phases of the model are subject to poisoning attacks and adversarial attacks, respectively. To address these severe security threats against the AI-based systems, in this article, we review the challenges and recent research advances for security issues in AI, so as to depict an overall blueprint for AI security. More specifically, we first take the lifecycle of an AI-based system as a guide to introduce the security threats that emerge at each stage, which is followed by a detailed summary for corresponding countermeasures. Finally, some of the future challenges and opportunities for the security issues in AI will also be discussed.

computer science, theory & methods
A Review of Artificial Intelligence in Security and Privacy: Research Advances, Applications, Opportunities, and Challenges

Yazan Alaya Al-Khassawneh

DOI: https://doi.org/10.17509/ijost.v8i1.52709

2022-09-20

Indonesian Journal of Science and Technology

Abstract:Artificial intelligence has the potential to address many societal, economic, and environmental challenges, but only if AI-enabled gadgets are kept secure. Many artificial intelligence (AI) models produced in recent years can be hacked by utilizing cutting-edge techniques. This issue has sparked intense research into adversarial AI to develop machine and deep learning models that can withstand various types of attacks. We provide a detailed summary of artificial intelligence in this paper to prove how adversarial attacks against AI applications can be mounted, covering topics such as confrontational knowledge and capabilities, existing methods for actually producing adversarial examples, and existing cyber defense models. In addition, we investigated numerous cyber countermeasures that could defend AI applications against these attacks and offered a systematic approach for demonstrating war strategies against machine learning and artificial intelligence. To safeguard AI applications, we emphasize the importance of understanding the intentions and methods of possible attackers. In the end, we list the biggest problems and most interesting research areas in the field of AI privacy and security.
Security and Privacy for Artificial Intelligence: Opportunities and Challenges

Ayodeji Oseni,Nour Moustafa,Helge Janicke,Peng Liu,Zahir Tari,Athanasios Vasilakos

DOI: https://doi.org/10.48550/arXiv.2102.04661

2021-02-09

Cryptography and Security

Abstract:The increased adoption of Artificial Intelligence (AI) presents an opportunity to solve many socio-economic and environmental challenges; however, this cannot happen without securing AI-enabled technologies. In recent years, most AI models are vulnerable to advanced and sophisticated hacking techniques. This challenge has motivated concerted research efforts into adversarial AI, with the aim of developing robust machine and deep learning models that are resilient to different types of adversarial scenarios. In this paper, we present a holistic cyber security review that demonstrates adversarial attacks against AI applications, including aspects such as adversarial knowledge and capabilities, as well as existing methods for generating adversarial examples and existing cyber defence models. We explain mathematical AI models, especially new variants of reinforcement and federated learning, to demonstrate how attack vectors would exploit vulnerabilities of AI models. We also propose a systematic framework for demonstrating attack techniques against AI applications and reviewed several cyber defences that would protect AI applications against those attacks. We also highlight the importance of understanding the adversarial goals and their capabilities, especially the recent attacks against industry applications, to develop adaptive defences that assess to secure AI applications. Finally, we describe the main challenges and future research directions in the domain of security and privacy of AI technologies.
Safeguarding Mail-Order DNA Synthesis in the Age of Artificial Intelligence

Stephanie Batalis,Caroline Schuerger,Gigi Kwik Gronvall,Matthew E Walsh

DOI: https://doi.org/10.1089/apb.2023.0020

2024-06-20

Abstract:Introduction: Artificial intelligence (AI) tools continue to be developed and used within the life sciences. The impact of these tools on the biosecurity landscape surrounding mail-order DNA synthesis and how to address the impacts have not been critically examined in the literature. Methods: The impacts of AI-driven chatbots and biological design tools on the biosecurity landscape surrounding mail-order DNA synthesis were analyzed and described. The findings are informed by the authors' experience in the field. Results: Generally, chatbots lower barriers to access of information that could be misused while biological design tools may provide new abilities to users with the intent of misuse. Six recommendations to the United States Government that attempt to maximize the benefits of these new technologies while mitigating risks are provided. Conclusion: Mandating mail-order DNA synthesis providers to screen DNA synthesis orders is a critical safeguarding step that should be taken as soon as possible. Over time, biological design tools will reduce the effectiveness of such a regulation and actions should be taken now to limit the negative impacts in the future.
Biosecurity Risk Assessment for the Use of Artificial Intelligence in Synthetic Biology

Leyma P De Haro

DOI: https://doi.org/10.1089/apb.2023.0031

2024-06-20

Abstract:Background: The integration of Artificial Intelligence (AI) with synthetic biology is driving unprecedented progress in both fields. However, this integration introduces complex biosecurity challenges. Addressing these concerns, this article proposes a specialized biosecurity risk assessment process designed to evaluate the incorporation of AI in synthetic biology. Methods: A set of tailored tools and methodology was developed for conducting biosecurity risk assessments of AI language models used for synthetic biology. These resources were developed to guide risk management professionals through a systematic process of identifying, evaluating, and mitigating potential risks. Results: The tools and methodology provided offer a structured approach to risk assessment, enabling risk management professionals to comprehensively analyze the biosecurity implications of AI applications in synthetic biology. They facilitate the identification of potential risks and the development of effective mitigation strategies. An example of a risk assessment performed on the large language model "ChatGPT 4.0" is provided here. Conclusion: AI's role in synthetic biology is rapidly expanding; thus, establishing proactive and secure practices is crucial. The biosecurity risk assessment tools and methodology presented here are the first provided in the literature and will be instrumental steps toward the responsible integration of AI in synthetic biology. By adopting these resources, the biorisk management community can effectively navigate and manage the biosecurity challenges posed by AI, ensuring its responsible and secure application in the field of synthetic biology.
Virtual Labs and Designer Bugs - Generative AI, Synthetic Biology and National Security

Brendan Walker-Munro

Abstract:AI technologies can pose a major national security concern. AI programs could be used to develop chemical and biological agents which circumvent existing protective measures or medical treatments, or to design pathogens with capabilities they do not naturally possess (gain-of-function research). Although Australia has a strong legislative framework relating to research into genetically modified organisms, the framework requires the interaction of more than 10 different government departments, universities and funding agencies. Further, there are few guidelines about the responsible use of AI in biological research where existing laws and policies do not apply to research that is conducted "virtually", even where that research may have national security implications. This article explores these under-scrutinised concepts in Australia's biological security frameworks.
Artificial intelligence and biological misuse: Differentiating risks of language models and biological design tools

Jonas B. Sandbrink

2023-12-23

Abstract:As advancements in artificial intelligence (AI) propel progress in the life sciences, they may also enable the weaponisation and misuse of biological agents. This article differentiates two classes of AI tools that could pose such biosecurity risks: large language models (LLMs) and biological design tools (BDTs). LLMs, such as GPT-4 and its successors, might provide dual-use information and thus remove some barriers encountered by historical biological weapons efforts. As LLMs are turned into multi-modal lab assistants and autonomous science tools, this will increase their ability to support non-experts in performing laboratory work. Thus, LLMs may in particular lower barriers to biological misuse. In contrast, BDTs will expand the capabilities of sophisticated actors. Concretely, BDTs may enable the creation of pandemic pathogens substantially worse than anything seen to date and could enable forms of more predictable and targeted biological weapons. In combination, the convergence of LLMs and BDTs could raise the ceiling of harm from biological agents and could make them broadly accessible. A range of interventions would help to manage risks. Independent pre-release evaluations could help understand the capabilities of models and the effectiveness of safeguards. Options for differentiated access to such tools should be carefully weighed with the benefits of openly releasing systems. Lastly, essential for mitigating risks will be universal and enhanced screening of gene synthesis products.

Computers and Society
Control Risk for Potential Misuse of Artificial Intelligence in Science

Jiyan He,Weitao Feng,Yaosen Min,Jingwei Yi,Kunsheng Tang,Shuai Li,Jie Zhang,Kejiang Chen,Wenbo Zhou,Xing Xie,Weiming Zhang,Nenghai Yu,Shuxin Zheng

DOI: https://doi.org/10.48550/arXiv.2312.06632

2023-12-12

Abstract:The expanding application of Artificial Intelligence (AI) in scientific fields presents unprecedented opportunities for discovery and innovation. However, this growth is not without risks. AI models in science, if misused, can amplify risks like creation of harmful substances, or circumvention of established regulations. In this study, we aim to raise awareness of the dangers of AI misuse in science, and call for responsible AI development and use in this domain. We first itemize the risks posed by AI in scientific contexts, then demonstrate the risks by highlighting real-world examples of misuse in chemical science. These instances underscore the need for effective risk management strategies. In response, we propose a system called SciGuard to control misuse risks for AI models in science. We also propose a red-teaming benchmark SciMT-Safety to assess the safety of different systems. Our proposed SciGuard shows the least harmful impact in the assessment without compromising performance in benign tests. Finally, we highlight the need for a multidisciplinary and collaborative effort to ensure the safe and ethical use of AI models in science. We hope that our study can spark productive discussions on using AI ethically in science among researchers, practitioners, policymakers, and the public, to maximize benefits and minimize the risks of misuse.

Artificial Intelligence
Harnessing artificial intelligence (AI) for cybersecurity: Challenges, opportunities, risks, future directions

Zarif Bin Akhtar,Ahmed Tajbiul Rawol

DOI: https://doi.org/10.59400/cai.v2i2.1485

2024-10-25

Abstract:The integration of artificial intelligence (AI) into cybersecurity has brought about transformative advancements in threat detection and mitigation, yet it also introduces new vulnerabilities and potential threats. This research exploration systematically investigates the critical issues surrounding AI within cybersecurity, focusing on specific vulnerabilities and the potential for AI systems to be exploited by malicious actors. The research aims to address these challenges by swotting and analyzing existing methodologies designed to mitigate such risks. Through a detailed exploration of modern scientific research, this manuscript identifies the dual-edged impact of AI on cybersecurity, emphasizing both the opportunities and the dangers. The findings highlight the need for strategic solutions that not only enhance digital security and user privacy but also address the ethical and regulatory aspects of AI in cybersecurity. Key contributions include a comprehensive analysis of emerging trends, challenges, and the development of AI-driven cybersecurity frameworks. The research also provides actionable recommendations for the future development of robust, reliable, and secure AI-based systems, bridging current knowledge gaps and offering valuable insights for academia and industry alike.
International Terrorism and Social Threats of Artificial Intelligence

Yaser Esmailzadeh,Ebrahim Motaghi

DOI: https://doi.org/10.30884/jogs/2024.01.09

2024-05-30

Journal of Globalization Studies

Abstract:This article delves into the link between global terrorism and the growing dangers presented by Artificial Intelligence (AI). We examine how terrorism utilizes AI technologies, such as advanced deep learning algorithms like ChatGPT, to bolster their activities both on the internet and in the world. Additionally, we assess the possible advantages and obstacles of using AI to combat terrorism, emphasizing the ethical and legal dilemmas involved. The article discusses the importance of regulating, educating, and prioritizing ethical considerations in AI development to tackle issues like disinformation, privacy violations, and job displacement. It emphasizes the need for a comprehensive approach involving cooperation among various groups to tackle the challenges posed by AI-driven terrorism while advocating for human rights and social justice.
Security Threats in Agentic AI System

Raihan Khan,Sayak Sarkar,Sainik Kumar Mahata,Edwin Jose

2024-10-16

Abstract:This research paper explores the privacy and security threats posed to an Agentic AI system with direct access to database systems. Such access introduces significant risks, including unauthorized retrieval of sensitive information, potential exploitation of system vulnerabilities, and misuse of personal or confidential data. The complexity of AI systems combined with their ability to process and analyze large volumes of data increases the chances of data leaks or breaches, which could occur unintentionally or through adversarial manipulation. Furthermore, as AI agents evolve with greater autonomy, their capacity to bypass or exploit security measures becomes a growing concern, heightening the need to address these critical vulnerabilities in agentic systems.

Cryptography and Security,Multiagent Systems
The Convergence of AI and Synthetic Biology: The Looming Deluge

Cindy Vindman,Benjamin Trump,Christopher Cummings,Madison Smith,Alexander J. Titus,Ken Oye,Valentina Prado,Eyup Turmus,Igor Linkov

2024-04-29

Abstract:The convergence of artificial intelligence (AI) and synthetic biology is rapidly accelerating the pace of biological discovery and engineering. AI techniques, such as large language models and biological design tools, are enabling the automated design, build, test, and learning cycles for engineered biological systems. This convergence promises to democratize synthetic biology and unlock novel applications across domains from medicine to environmental sustainability. However, it also poses significant risks around reliability, dual use, and governance. The opacity of AI models, the deskilling of workforces, and the outdated nature of current regulatory frameworks present challenges in ensuring responsible development. Urgent attention is needed to update governance structures, integrate human oversight into increasingly automated workflows, and foster a culture of responsibility among the growing community of bioengineers. Only by proactively addressing these issues can we realize the transformative potential of AI-driven synthetic biology while mitigating its risks.

Other Quantitative Biology
SECURE BY INTELLIGENCE: ENHANCING PRODUCTS WITH AI-DRIVEN SECURITY MEASURES

Sakthiswaran Rangaraju

DOI: https://doi.org/10.53555/ephijse.v9i3.212

2023-12-01

Abstract:In an increasingly interconnected digital landscape, the proliferation of sophisticated cyber threats poses significant challenges to the security and integrity of products and services. As traditional security measures struggle to keep pace with evolving threats, there exists a pressing need for innovative and adaptive approaches to safeguarding digital assets. This abstract introduces the concept of "Secure by Intelligence," a paradigm shift in product security that leverages the power of Artificial Intelligence (AI) to fortify defenses and proactively mitigate risks. This paper explores the integration of AI-driven security measures as a foundational element in enhancing the resilience of various products across industries. It delves into the core principles of AI-powered security, emphasizing the utilization of machine learning, deep learning, natural language processing, and anomaly detection to predict, detect, and respond to potential threats in real time. The key focus areas include Dynamic Threat Detection and Prediction, Behavioral Analysis and Anomaly Detection, and Automated Response and Adaptation: Through AI-based automation, systems can autonomously respond to security incidents, mitigating risks in real-time. Furthermore, adaptive AI systems learn from each encounter, enhancing their ability to preempt future attacks. Privacy-Preserving Solutions, Cross-Industry Applications. This paper illustrates real-world case studies and implementations where AI-driven security measures have significantly bolstered product security and resilience. It highlights the tangible advantages of adopting AI-centric security solutions, including improved threat detection accuracy, reduced response times, and enhanced adaptability to emerging cyber threats.

Security challenges by AI-assisted protein design

Protein design meets biosecurity

Could AI-designed proteins be weaponized? Scientists lay out safety guidelines

Toward AI-Resilient Screening of Nucleic Acid Synthesis Orders: Process, Results, and Recommendations

Threats and Opportunities with AI-based Cyber Security Intrusion Detection: A Review

Dual use of artificial-intelligence-powered drug discovery

Peptide Toxins as Biothreats and the Potential for AI Systems to Enhance Biosecurity

Artificial intelligence challenges in the face of biological threats: emerging catastrophic risks for public health

Artificial Intelligence Security: Threats and Countermeasures

A Review of Artificial Intelligence in Security and Privacy: Research Advances, Applications, Opportunities, and Challenges

Security and Privacy for Artificial Intelligence: Opportunities and Challenges

Safeguarding Mail-Order DNA Synthesis in the Age of Artificial Intelligence

Biosecurity Risk Assessment for the Use of Artificial Intelligence in Synthetic Biology

Virtual Labs and Designer Bugs - Generative AI, Synthetic Biology and National Security

Artificial intelligence and biological misuse: Differentiating risks of language models and biological design tools

Control Risk for Potential Misuse of Artificial Intelligence in Science

Harnessing artificial intelligence (AI) for cybersecurity: Challenges, opportunities, risks, future directions

International Terrorism and Social Threats of Artificial Intelligence

Security Threats in Agentic AI System

The Convergence of AI and Synthetic Biology: The Looming Deluge

SECURE BY INTELLIGENCE: ENHANCING PRODUCTS WITH AI-DRIVEN SECURITY MEASURES