AI-Driven Network Security and Privacy

Yu-an Tan,Qikun Zhang,Yuanzhang Li,Xiao Yu

DOI: https://doi.org/10.3390/electronics13122311

IF: 2.9

2024-06-14

Electronics

Abstract:While creating unprecedented opportunities, artificial intelligence is also accompanied by unprecedented risks [...]

engineering, electrical & electronic,physics, applied,computer science, information systems

What problem does this paper attempt to address?

Influence of AI on Network Security and Its Future Development

Xianni Zhang,Ziao Gao,Cong Cheng,Mengze Jia,Yuqi Liu

DOI: https://doi.org/10.1109/ICEICT61637.2024.10671232

2024-07-31

Abstract:With the rapid development of information technology, network security has become the focus of global attention. Hacking attacks, data leakage, network fraud and other threats emerge in an endless stream, bringing huge risks and losses to individuals, enterprises and countries. AI, as a powerful technical tool, Artificial Intelligence (AI) is gradually integrated into the field of network security, providing new ideas and methods for the defense and attack of network security. AI's learning and reasoning capabilities enable it to quickly identify and respond to complex network attacks, providing unprecedented protection capabilities for network security.

Engineering,Computer Science
Threats and Opportunities with AI-based Cyber Security Intrusion Detection: A Review

Bibhu Dash,Meraj Farheen Ansari,Pawankumar Sharma,Azad Ali

DOI: https://doi.org/10.5121/ijsea.2022.13502

2022-09-30

International Journal of Software Engineering & Applications

Abstract:Internet usage has increased quickly, particularly in the previous decade. With the widespread use of the internet, cybercrime is growing at an alarming rate in our daily lives. However, with the growth of artificial intelligence (AI), businesses are concentrating more on preventing cybercrime. AI is becoming an essential component of every business, affecting individuals worldwide. Cybercrime is one of the most prominent domains where AI has begun demonstrating valuable inputs. As a result, AI is being deployed as the first line of defense in most firms' systems. Because AI can detect new assaults faster than humans, it is the best alternative for constructing better protection against cybercrime. AI technologies also offer more significant potential in the development of such technology. This paper discusses recent cyber intrusions and how the AI-enabled industry is preparing to defend itself in the long run.
Social Risks in the Era of Generative AI

Xiaozhong Liu,Yu‐Ru Lin,Zhuoren Jiang,Qunfang Wu

DOI: https://doi.org/10.1002/pra2.1103

2024-01-01

Proceedings of the Association for Information Science and Technology

Abstract:ABSTRACTGenerative AI (GAI) technologies have demonstrated human‐level performance on a vast spectrum of tasks. However, recent studies have also delved into the potential threats and vulnerabilities posed by GAI, particularly as they become increasingly prevalent in sensitive domains such as elections and education. Their use in politics raises concerns about manipulation and misinformation. Further exploration is imperative to comprehend the social risks associated with GAI across diverse societal contexts. In this panel, we aim to dissect the impact and risks posed by GAI on our social fabric, examining both technological and societal perspectives. Additionally, we will present our latest investigations, including the manipulation of ideologies using large language models (LLMs), the potential risk of AI self‐consciousness, the application of Explainable AI (XAI) to identify patterns of misinformation and mitigate their dissemination, as well as the influence of GAI on the quality of public discourse. These insights will serve as catalysts for stimulating discussions among the audience on this crucial subject matter, and contribute to fostering a deeper understanding of the importance of responsible development and deployment of GAI technologies.
Artificial Intelligence Security: Threats and Countermeasures

Yupeng Hu,Wenxin Kuang,Zheng Qin,Kenli Li,Jiliang Zhang,Yansong Gao,Wenjia Li,Keqin Li

DOI: https://doi.org/10.1145/3487890

IF: 16.6

2023-01-31

ACM Computing Surveys

Abstract:In recent years, with rapid technological advancement in both computing hardware and algorithm, Artificial Intelligence (AI) has demonstrated significant advantage over human being in a wide range of fields, such as image recognition, education, autonomous vehicles, finance, and medical diagnosis. However, AI-based systems are generally vulnerable to various security threats throughout the whole process, ranging from the initial data collection and preparation to the training, inference, and final deployment. In an AI-based system, the data collection and pre-processing phase are vulnerable to sensor spoofing attacks and scaling attacks, respectively, while the training and inference phases of the model are subject to poisoning attacks and adversarial attacks, respectively. To address these severe security threats against the AI-based systems, in this article, we review the challenges and recent research advances for security issues in AI, so as to depict an overall blueprint for AI security. More specifically, we first take the lifecycle of an AI-based system as a guide to introduce the security threats that emerge at each stage, which is followed by a detailed summary for corresponding countermeasures. Finally, some of the future challenges and opportunities for the security issues in AI will also be discussed.

computer science, theory & methods
The Current Research Status of AI-Based Network Security Situational Awareness

Maoli Wang,Guangxue Song,Yang Yu,Bowen Zhang

DOI: https://doi.org/10.3390/electronics12102309

IF: 2.9

2023-05-20

Electronics

Abstract:Network security situational awareness is based on the extraction and analysis of big data, and by understanding these data to evaluate the current network security status and predict future development trends, provide feedback to decision-makers to make corresponding countermeasures, and achieve security protection for the network environment. This article focuses on artificial intelligence, summarizes the related definitions and classic models of network security situational awareness, and provides an overview of artificial intelligence. Starting from the method of machine learning, it specifically introduces the research status of neural-network-based network security situational awareness and summarizes the research work in recent years. Finally, the future development trends of network security situational awareness are summarized, and its prospects.

engineering, electrical & electronic,computer science, information systems,physics, applied
Security and Privacy for Artificial Intelligence: Opportunities and Challenges

Ayodeji Oseni,Nour Moustafa,Helge Janicke,Peng Liu,Zahir Tari,Athanasios Vasilakos

DOI: https://doi.org/10.48550/arXiv.2102.04661

2021-02-09

Cryptography and Security

Abstract:The increased adoption of Artificial Intelligence (AI) presents an opportunity to solve many socio-economic and environmental challenges; however, this cannot happen without securing AI-enabled technologies. In recent years, most AI models are vulnerable to advanced and sophisticated hacking techniques. This challenge has motivated concerted research efforts into adversarial AI, with the aim of developing robust machine and deep learning models that are resilient to different types of adversarial scenarios. In this paper, we present a holistic cyber security review that demonstrates adversarial attacks against AI applications, including aspects such as adversarial knowledge and capabilities, as well as existing methods for generating adversarial examples and existing cyber defence models. We explain mathematical AI models, especially new variants of reinforcement and federated learning, to demonstrate how attack vectors would exploit vulnerabilities of AI models. We also propose a systematic framework for demonstrating attack techniques against AI applications and reviewed several cyber defences that would protect AI applications against those attacks. We also highlight the importance of understanding the adversarial goals and their capabilities, especially the recent attacks against industry applications, to develop adaptive defences that assess to secure AI applications. Finally, we describe the main challenges and future research directions in the domain of security and privacy of AI technologies.
Artificial intelligence and machine learning in dynamic cyber risk analytics at the edge

Petar Radanliev,David De Roure,Rob Walton,Max Van Kleek,Rafael Mantilla Montalvo,La’Treall Maddox,Omar Santos,Peter Burnap,Eirini Anthi

DOI: https://doi.org/10.1007/s42452-020-03559-4

2020-10-06

SN Applied Sciences

Abstract:Abstract We explore the potential and practical challenges in the use of artificial intelligence (AI) in cyber risk analytics, for improving organisational resilience and understanding cyber risk. The research is focused on identifying the role of AI in connected devices such as Internet of Things (IoT) devices. Through literature review, we identify wide ranging and creative methodologies for cyber analytics and explore the risks of deliberately influencing or disrupting behaviours to socio-technical systems. This resulted in the modelling of the connections and interdependencies between a system's edge components to both external and internal services and systems. We focus on proposals for models, infrastructures and frameworks of IoT systems found in both business reports and technical papers. We analyse this juxtaposition of related systems and technologies, in academic and industry papers published in the past 10 years. Then, we report the results of a qualitative empirical study that correlates the academic literature with key technological advances in connected devices. The work is based on grouping future and present techniques and presenting the results through a new conceptual framework. With the application of social science's grounded theory, the framework details a new process for a prototype of AI-enabled dynamic cyber risk analytics at the edge.
AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathways

Zehang Deng,Yongjian Guo,Changzhou Han,Wanlun Ma,Junwu Xiong,Sheng Wen,Yang Xiang

2024-09-06

Abstract:An Artificial Intelligence (AI) agent is a software entity that autonomously performs tasks or makes decisions based on pre-defined objectives and data inputs. AI agents, capable of perceiving user inputs, reasoning and planning tasks, and executing actions, have seen remarkable advancements in algorithm development and task performance. However, the security challenges they pose remain under-explored and unresolved. This survey delves into the emerging security threats faced by AI agents, categorizing them into four critical knowledge gaps: unpredictability of multi-step user inputs, complexity in internal executions, variability of operational environments, and interactions with untrusted external entities. By systematically reviewing these threats, this paper highlights both the progress made and the existing limitations in safeguarding AI agents. The insights provided aim to inspire further research into addressing the security threats associated with AI agents, thereby fostering the development of more robust and secure AI agent applications.

Cryptography and Security,Artificial Intelligence
Intelligent networking in adversarial environment: challenges and opportunities

Yi Zhao,Ke Xu,Qi Li,Haiyang Wang,Dan Wang,Min Zhu

DOI: https://doi.org/10.1007/s11432-021-3463-9

2022-01-01

Science China Information Sciences

Abstract:Although deep learning technologies have been widely exploited in many fields, they are vulnerable to adversarial attacks by adding small perturbations to legitimate inputs to fool targeted models. However, few studies have focused on intelligent networking in such an adversarial environment, which can pose serious security threats. In fact, while challenging intelligent networking, adversarial environments also bring about opportunities. In this paper, we, for the first time, simultaneously analyze the challenges and opportunities that the adversarial environment brings to intelligent networking. Specifically, we focus on challenges that the adversarial environment will pose on the existing intelligent networking. Furthermore, we investigate frameworks and approaches that combine adversarial machine learning with intelligent networking to solve the existing deficiencies of intelligent networking. Finally, we summarize the issues, including opportunities and challenges, which can allow researchers to focus on intelligent networking in adversarial environments.
Artificial intelligence in cyber security: research advances, challenges, and opportunities

Zhimin Zhang,Huansheng Ning,Feifei Shi,Fadi Farha,Yang Xu,Jiabo Xu,Fan Zhang,Kim-Kwang Raymond Choo

DOI: https://doi.org/10.1007/s10462-021-09976-0

IF: 9.588

2021-03-13

Artificial Intelligence Review

Abstract:In recent times, there have been attempts to leverage artificial intelligence (AI) techniques in a broad range of cyber security applications. Therefore, this paper surveys the existing literature (comprising 54 papers mainly published between 2016 and 2020) on the applications of AI in user access authentication, network situation awareness, dangerous behavior monitoring, and abnormal traffic identification. This paper also identifies a number of limitations and challenges, and based on the findings, a conceptual human-in-the-loop intelligence cyber security model is presented.

computer science, artificial intelligence
AI-Driven Anonymization: Protecting Personal Data Privacy While Leveraging Machine Learning

Le Yang,Miao Tian,Duan Xin,Qishuo Cheng,Jiajian Zheng

2024-02-27

Abstract:The development of artificial intelligence has significantly transformed people's lives. However, it has also posed a significant threat to privacy and security, with numerous instances of personal information being exposed online and reports of criminal attacks and theft. Consequently, the need to achieve intelligent protection of personal information through machine learning algorithms has become a paramount concern. Artificial intelligence leverages advanced algorithms and technologies to effectively encrypt and anonymize personal data, enabling valuable data analysis and utilization while safeguarding privacy. This paper focuses on personal data privacy protection and the promotion of anonymity as its core research objectives. It achieves personal data privacy protection and detection through the use of machine learning's differential privacy protection algorithm. The paper also addresses existing challenges in machine learning related to privacy and personal data protection, offers improvement suggestions, and analyzes factors impacting datasets to enable timely personal data privacy detection and protection.

Cryptography and Security,Artificial Intelligence,Machine Learning
Securing the Future: Exploring Privacy Risks and Security Questions in Robotic Systems

Diba Afroze,Yazhou Tu,Xiali Hei

DOI: https://doi.org/10.1007/978-3-031-51630-6_10

2024-09-16

Abstract:The integration of artificial intelligence, especially large language models in robotics, has led to rapid advancements in the field. We are now observing an unprecedented surge in the use of robots in our daily lives. The development and continual improvements of robots are moving at an astonishing pace. Although these remarkable improvements facilitate and enhance our lives, several security and privacy concerns have not been resolved yet. Therefore, it has become crucial to address the privacy and security threats of robotic systems while improving our experiences. In this paper, we aim to present existing applications and threats of robotics, anticipated future evolution, and the security and privacy issues they may imply. We present a series of open questions for researchers and practitioners to explore further.

Robotics,Human-Computer Interaction
Security Risks Concerns of Generative AI in the IoT

Honghui Xu,Yingshu Li,Olusesi Balogun,Shaoen Wu,Yue Wang,Zhipeng Cai

2024-03-30

Abstract:In an era where the Internet of Things (IoT) intersects increasingly with generative Artificial Intelligence (AI), this article scrutinizes the emergent security risks inherent in this integration. We explore how generative AI drives innovation in IoT and we analyze the potential for data breaches when using generative AI and the misuse of generative AI technologies in IoT ecosystems. These risks not only threaten the privacy and efficiency of IoT systems but also pose broader implications for trust and safety in AI-driven environments. The discussion in this article extends to strategic approaches for mitigating these risks, including the development of robust security protocols, the multi-layered security approaches, and the adoption of AI technological solutions. Through a comprehensive analysis, this article aims to shed light on the critical balance between embracing AI advancements and ensuring stringent security in IoT, providing insights into the future direction of these intertwined technologies.

Cryptography and Security,Artificial Intelligence
Survey of Artificial Intelligence Data Security and Privacy Protection

REN Kui,MENG Quanrun,YAN Shoukun,QIN Zhan

DOI: https://doi.org/10.11959/j.issn.2096-109x.2021001

2021-01-01

Abstract:Artificial intelligence and deep learning algorithms are developing rapidly. These emerging techniques have been widely used in audio and video recognition, natural language processing and other fields. However, in recent years, researchers have found that there are many security risks in the current mainstream artificial intelligence model, and these problems will limit the development of AI. Therefore, the data security and privacy protection was studied in AI. For data and privacy leakage, the model output based and model update based problem of data leakage were studied. In the model output based problem of data leakage, the principles and research status of model extraction attack, model inversion attack and membership inference attack were discussed. In the model update based problem of data leakage, how attackers steal private data in the process of distributed training was discussed. For data and privacy protection, three kinds of defense methods, namely model structure defense, information confusion defense and query control defense were studied. In summarize, the theoretical foundations, classic algorithms of data inference attack techniques were introduced. A few research efforts on the defense techniques were described in order to provoke further research efforts in this critical area.
Some discussions on critical information security issues in the artificial intelligence era

Vuong Quan Hoang,Viet-Phuong La,Hong-Son Nguyen,Minh-Hoang Nguyen

DOI: https://doi.org/10.1007/s00146-024-02023-w

2024-07-26

AI & Society

Abstract:The rapid advancement of information technology platforms and programming languages has transformed the dynamics and development of human society. The cyberspace and associated utilities are expanding, leading to a gradual shift from real-world living to virtual life (also known as cyberspace or digital space). The expansion and development of natural language processing models and large language models demonstrate human-like characteristics in reasoning, perception, attention, and creativity, helping humans overcome operational barriers. Alongside the immense potential of artificial intelligence (AI) are new security loopholes and more complex information security risks. As society is still trying to transition to a new phase to adapt to technological changes, the AI revolution continues to unfold, necessitating a reconsideration of the trajectory of societal transformation as it could exacerbate the aforementioned information security risks. Specifically, how should society evolve to keep pace with the transformative impact of the current AI technology wave? How can we manage and harness their power while ensuring information security as our presence in the virtual world increases? This article aims to shed light on and address these questions.
AI Risk Management Should Incorporate Both Safety and Security

Xiangyu Qi,Yangsibo Huang,Yi Zeng,Edoardo Debenedetti,Jonas Geiping,Luxi He,Kaixuan Huang,Udari Madhushani,Vikash Sehwag,Weijia Shi,Boyi Wei,Tinghao Xie,Danqi Chen,Pin-Yu Chen,Jeffrey Ding,Ruoxi Jia,Jiaqi Ma,Arvind Narayanan,Weijie J Su,Mengdi Wang,Chaowei Xiao,Bo Li,Dawn Song,Peter Henderson,Prateek Mittal

2024-05-30

Abstract:The exposure of security vulnerabilities in safety-aligned language models, e.g., susceptibility to adversarial attacks, has shed light on the intricate interplay between AI safety and AI security. Although the two disciplines now come together under the overarching goal of AI risk management, they have historically evolved separately, giving rise to differing perspectives. Therefore, in this paper, we advocate that stakeholders in AI risk management should be aware of the nuances, synergies, and interplay between safety and security, and unambiguously take into account the perspectives of both disciplines in order to devise mostly effective and holistic risk mitigation approaches. Unfortunately, this vision is often obfuscated, as the definitions of the basic concepts of "safety" and "security" themselves are often inconsistent and lack consensus across communities. With AI risk management being increasingly cross-disciplinary, this issue is particularly salient. In light of this conceptual challenge, we introduce a unified reference framework to clarify the differences and interplay between AI safety and AI security, aiming to facilitate a shared understanding and effective collaboration across communities.

Cryptography and Security,Artificial Intelligence
Managing extreme AI risks amid rapid progress

Yoshua Bengio,Geoffrey Hinton,Andrew Yao,Dawn Song,Pieter Abbeel,Trevor Darrell,Yuval Noah Harari,Ya-Qin Zhang,Lan Xue,Shai Shalev-Shwartz,Gillian Hadfield,Jeff Clune,Tegan Maharaj,Frank Hutter,Atılım Güneş Baydin,Sheila McIlraith,Qiqi Gao,Ashwin Acharya,David Krueger,Anca Dragan,Philip Torr,Stuart Russell,Daniel Kahneman,Jan Brauner,Sören Mindermann

DOI: https://doi.org/10.1126/science.adn0117

2024-05-23

Abstract:Artificial Intelligence (AI) is progressing rapidly, and companies are shifting their focus to developing generalist AI systems that can autonomously act and pursue goals. Increases in capabilities and autonomy may soon massively amplify AI's impact, with risks that include large-scale social harms, malicious uses, and an irreversible loss of human control over autonomous AI systems. Although researchers have warned of extreme risks from AI, there is a lack of consensus about how exactly such risks arise, and how to manage them. Society's response, despite promising first steps, is incommensurate with the possibility of rapid, transformative progress that is expected by many experts. AI safety research is lagging. Present governance initiatives lack the mechanisms and institutions to prevent misuse and recklessness, and barely address autonomous systems. In this short consensus paper, we describe extreme risks from upcoming, advanced AI systems. Drawing on lessons learned from other safety-critical technologies, we then outline a comprehensive plan combining technical research and development with proactive, adaptive governance mechanisms for a more commensurate preparation.

Computers and Society,Artificial Intelligence,Computation and Language,Machine Learning
Living with Artificial Intelligence: A Paradigm Shift toward Future Network Traffic Control.

Jun Xu,Kaishun Wu

DOI: https://doi.org/10.1109/MNET.2018.1800119

IF: 10.294

2018-01-01

IEEE Network

Abstract:Future Internet is expected to meet explosive traffic growth and extremely complex architecture, which tend to make the traditional NTC strategies inefficient and even ineffective. Inspired by the latest breakthroughs of AI and its power to address large-scale and complex difficulties, the network community has begun to consider shifting the NTC paradigm from legacy rule-based to novel AI-based. A...
The Security of Autonomous Driving: Threats, Defenses, and Future Directions

Kui Ren,Qian Wang,Cong Wang,Zhan Qin,Xiaodong Lin

DOI: https://doi.org/10.1109/jproc.2019.2948775

IF: 20.6

2019-01-01

Proceedings of the IEEE

Abstract:Autonomous vehicles (AVs) have promised to drastically improve the convenience of driving by releasing the burden of drivers and reducing traffic accidents with more precise control. With the fast development of artificial intelligence and significant advancements of the Internet of Things technologies, we have witnessed the steady progress of autonomous driving over the recent years. As promising as it is, the march of autonomous driving technologies also faces new challenges, among which security is the top concern. In this article, we give a systematic study on the security threats surrounding autonomous driving, from the angles of perception, navigation, and control. In addition to the in-depth overview of these threats, we also summarize the corresponding defense strategies. Furthermore, we discuss future research directions about the new security threats, especially those related to deep-learning-based self-driving vehicles. By providing the security guidelines at this early stage, we aim to promote new techniques and designs related to AVs from both academia and industry and boost the development of secure autonomous driving.
Balancing Transparency and Risk: The Security and Privacy Risks of Open-Source Machine Learning Models

Dominik Hintersdorf,Lukas Struppek,Kristian Kersting

DOI: https://doi.org/10.48550/arXiv.2308.09490

IF: 5.414

2023-08-18

Machine Learning

Abstract:The field of artificial intelligence (AI) has experienced remarkable progress in recent years, driven by the widespread adoption of open-source machine learning models in both research and industry. Considering the resource-intensive nature of training on vast datasets, many applications opt for models that have already been trained. Hence, a small number of key players undertake the responsibility of training and publicly releasing large pre-trained models, providing a crucial foundation for a wide range of applications. However, the adoption of these open-source models carries inherent privacy and security risks that are often overlooked. To provide a concrete example, an inconspicuous model may conceal hidden functionalities that, when triggered by specific input patterns, can manipulate the behavior of the system, such as instructing self-driving cars to ignore the presence of other vehicles. The implications of successful privacy and security attacks encompass a broad spectrum, ranging from relatively minor damage like service interruptions to highly alarming scenarios, including physical harm or the exposure of sensitive user data. In this work, we present a comprehensive overview of common privacy and security threats associated with the use of open-source models. By raising awareness of these dangers, we strive to promote the responsible and secure use of AI systems.

AI-Driven Network Security and Privacy

Influence of AI on Network Security and Its Future Development

Threats and Opportunities with AI-based Cyber Security Intrusion Detection: A Review

Social Risks in the Era of Generative AI

Artificial Intelligence Security: Threats and Countermeasures

The Current Research Status of AI-Based Network Security Situational Awareness

Security and Privacy for Artificial Intelligence: Opportunities and Challenges

Artificial intelligence and machine learning in dynamic cyber risk analytics at the edge

AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathways

Intelligent networking in adversarial environment: challenges and opportunities

Artificial intelligence in cyber security: research advances, challenges, and opportunities

AI-Driven Anonymization: Protecting Personal Data Privacy While Leveraging Machine Learning

Securing the Future: Exploring Privacy Risks and Security Questions in Robotic Systems

Security Risks Concerns of Generative AI in the IoT

Survey of Artificial Intelligence Data Security and Privacy Protection

Some discussions on critical information security issues in the artificial intelligence era

AI Risk Management Should Incorporate Both Safety and Security

Managing extreme AI risks amid rapid progress

Living with Artificial Intelligence: A Paradigm Shift toward Future Network Traffic Control.

The Security of Autonomous Driving: Threats, Defenses, and Future Directions

Balancing Transparency and Risk: The Security and Privacy Risks of Open-Source Machine Learning Models