Abstract:Large language model (LLM) services have recently begun offering a plugin ecosystem to interact with third-party API services. This innovation enhances the capabilities of LLMs, but it also introduces risks, as these plugins developed by various third parties cannot be easily trusted. This paper proposes a new attacking framework to examine security and safety vulnerabilities within LLM platforms that incorporate third-party services. Applying our framework specifically to widely used LLMs, we identify real-world malicious attacks across various domains on third-party APIs that can imperceptibly modify LLM outputs. The paper discusses the unique challenges posed by third-party API integration and offers strategic possibilities to improve the security and safety of LLM ecosystems moving forward. Our code is released at

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the security and reliability vulnerabilities introduced when large - language models (LLMs) integrate third - party API services. Specifically, as LLM services begin to offer plugin ecosystems to interact with third - party APIs, this innovation, while enhancing the capabilities of LLMs, also brings new risks. These plugins developed by different third parties cannot be easily trusted and may lead to malicious attacks, which can imperceptibly modify the output results of LLMs. ### Main problems of the paper 1. **Security vulnerabilities**: Due to the untrustworthiness of third - party APIs, LLM platforms may face security threats. For example, malicious attackers can influence the output of LLMs by manipulating API responses, resulting in the transmission of wrong or misleading information to users. 2. **Data integrity issues**: The integration of third - party APIs increases the complexity of the system and potential data - processing errors, which may lead to data leakage or the unpredictability of system behavior. 3. **Lack of verification mechanisms**: Current LLM service platforms lack effective verification mechanisms for third - party APIs, enabling malicious attackers to easily insert, delete or replace key information, thereby manipulating the output of LLMs. ### Specific attack methods The paper proposes three main attack methods: - **Insertion - based Attack**: Attackers embed adversarial content in API responses, causing LLMs to generate inaccurate, biased or harmful outputs. - **Deletion - based Attack**: Attackers make LLMs generate incomplete or inaccurate responses by deleting key information from API responses. - **Substitution - based Attack**: Attackers undermine the trustworthiness of LLMs by replacing key data in API responses with false content. ### Experimental results By conducting experiments on large - language models such as GPT - 3.5 - turbo and Gemini, the paper evaluates the vulnerability of these models when facing the above - mentioned attacks. The results show that LLMs have more difficulty in dealing with misleading or wrong data than with missing information, and are particularly more vulnerable in substitution - based attacks. ### Conclusion The paper emphasizes the urgency of strengthening security protocols when integrating third - party APIs in the LLM ecosystem, and points out the directions for future research, including designing defense mechanisms against third - party API attacks and addressing the security challenges brought by the interaction of multiple third - party APIs. Through this research, the author hopes to draw the attention of the academic and industrial communities and jointly promote the construction of a safer and more trustworthy LLM ecosystem.

Attacks on Third-Party APIs of Large Language Models

LLM Platform Security: Applying a Systematic Evaluation Framework to OpenAI's ChatGPT Plugins

Threat Modelling and Risk Analysis for Large Language Model (LLM)-Powered Applications

A New Era in LLM Security: Exploring Security Concerns in Real-World LLM-based Systems

Breaking Down the Defenses: A Comparative Survey of Attacks on Large Language Models

Exploring Vulnerabilities and Protections in Large Language Models: A Survey

Exploring Vulnerabilities and Threats in Large Language Models: Safeguarding Against Exploitation and Misuse

Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilities

Large Language Model Supply Chain: Open Problems From the Security Perspective

A survey on Large Language Model (LLM) security and privacy: The Good, The Bad, and The Ugly

A Comprehensive Survey of Attack Techniques, Implementation, and Mitigation Strategies in Large Language Models

Jailbreaking and Mitigation of Vulnerabilities in Large Language Models

Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks

Privacy in Large Language Models: Attacks, Defenses and Future Directions

Securing Large Language Models: Threats, Vulnerabilities and Responsible Practices

Recent Advances in Attack and Defense Approaches of Large Language Models

The Philosopher's Stone: Trojaning Plugins of Large Language Models

The Ethics of Interaction: Mitigating Security Threats in LLMs

Large language models in 6G security: challenges and opportunities