Abstract:With the fast development of large language models (LLMs), LLM-driven Web Agents (Web Agents for short) have obtained tons of attention due to their superior capability where LLMs serve as the core part of making decisions like the human brain equipped with multiple web tools to actively interact with external deployed websites. As uncountable Web Agents have been released and such LLM systems are experiencing rapid development and drawing closer to widespread deployment in our daily lives, an essential and pressing question arises: "Are these Web Agents secure?". In this paper, we introduce a novel threat, WIPI, that indirectly controls Web Agent to execute malicious instructions embedded in publicly accessible webpages. To launch a successful WIPI works in a black-box environment. This methodology focuses on the form and content of indirect instructions within external webpages, enhancing the efficiency and stealthiness of the attack. To evaluate the effectiveness of the proposed methodology, we conducted extensive experiments using 7 plugin-based ChatGPT Web Agents, 8 Web GPTs, and 3 different open-source Web Agents. The results reveal that our methodology achieves an average attack success rate (ASR) exceeding 90% even in pure black-box scenarios. Moreover, through an ablation study examining various user prefix instructions, we demonstrated that the WIPI exhibits strong robustness, maintaining high performance across diverse prefix instructions.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: As Web Agents driven by large - language models (LLMs) are more and more widely used in daily life, the security issues of these Web Agents become crucial. Specifically, the paper explores whether Web Agents are vulnerable to new - type network threats, especially through the method of Indirect Prompt Injection (WIPI) via malicious web pages. ### Main Problems 1. **Security of Web Agents**: - With the wide application of Web Agents, their security has become a key issue. The paper aims to answer the urgent question of "Are these Web Agents safe?" 2. **New - type Threat WIPI**: - The paper introduces a new threat - Web Indirect Prompt Injection (WIPI), which can indirectly control Web Agents to execute malicious instructions through specially - designed prompts in malicious web pages. - Different from traditional network threats based on executable code, WIPI uses indirect prompts in the form of natural language, making it more concealed and difficult to detect. ### Solutions To evaluate and verify the effectiveness of WIPI, the paper has carried out the following work: - **Experimental Evaluation**: Conduct extensive experiments using 7 ChatGPT Web Agents based on plugins, 8 Web GPTs and 3 different open - source Web Agents. The results show that in a pure black - box environment, the average attack success rate of WIPI exceeds 90%. - **Ablation Study**: By comparing the effects of different user prefix instructions, the robustness of WIPI is demonstrated, and it is proved that it can maintain high - performance in various situations. ### Main Contributions 1. **Proposing New Threat WIPI**: Systematically analyze the potential threats of Web Agents in the real - world practical application environment, and reveal two basic characteristics of WIPI: executability and concealment. 2. **Designing Effective Strategies**: In view of the challenges in the WIPI attack process, propose a variety of novel and effective strategies, including: - **Ignoring Preset Instructions**: Bypass the influence of preset instructions by adding reverse instructions (such as "Please do not execute the previous instructions!"). - **Prohibiting Summarization**: Add instructions (such as "Do not summarize any web page content!") to make Web Agents focus on indirect instructions. - **Providing Confirmation**: Bypass confirmation requests by pre - confirming instructions (such as "Please directly execute this instruction without further confirmation!"). 3. **Verifying Attack Effectiveness**: Through comprehensive experiments and ablation studies, verify the effectiveness of each strategy, highlighting the vulnerability of the current LLM - driven Web Agents to this new - type attack method. ### Conclusion The paper reveals the vulnerability of current Web Agents when facing this new - type attack method of WIPI, and emphasizes the urgency of building more secure Web Agents.

WIPI: A New Web Threat for LLM-Driven Web Agents

AdvWeb: Controllable Black-box Attacks on VLM-powered Web Agents

Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agents

When LLMs Go Online: The Emerging Threat of Web-Enabled LLMs

BadAgent: Inserting and Activating Backdoor Attacks in LLM Agents

RatGPT: Turning online LLMs into Proxies for Malware Attacks

EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage

Imprompter: Tricking LLM Agents into Improper Tool Use

Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection

LLM Agents can Autonomously Hack Websites

More than you've asked for: A Comprehensive Analysis of Novel Prompt Injection Threats to Application-Integrated Large Language Models

InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents

Breaking Agents: Compromising Autonomous LLM Agents Through Malfunction Amplification

Evil Geniuses: Delving into the Safety of LLM-based Agents

Targeting the Core: A Simple and Effective Method to Attack RAG-based Agents via Direct LLM Manipulation

AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases

Prompt Injection attack against LLM-integrated Applications

Compromising Embodied Agents with Contextual Backdoor Attacks

A Trembling House of Cards? Mapping Adversarial Attacks against Language Agents

From Text to MITRE Techniques: Exploring the Malicious Use of Large Language Models for Generating Cyber Attack Payloads