Abstract:Large language model (LLM) platforms, such as ChatGPT, have recently begun offering an app ecosystem to interface with third-party services on the internet. While these apps extend the capabilities of LLM platforms, they are developed by arbitrary third parties and thus cannot be implicitly trusted. Apps also interface with LLM platforms and users using natural language, which can have imprecise interpretations. In this paper, we propose a framework that lays a foundation for LLM platform designers to analyze and improve the security, privacy, and safety of current and future third-party integrated LLM platforms. Our framework is a formulation of an attack taxonomy that is developed by iteratively exploring how LLM platform stakeholders could leverage their capabilities and responsibilities to mount attacks against each other. As part of our iterative process, we apply our framework in the context of OpenAI's plugin (apps) ecosystem. We uncover plugins that concretely demonstrate the potential for the types of issues that we outline in our attack taxonomy. We conclude by discussing novel challenges and by providing recommendations to improve the security, privacy, and safety of present and future LLM-based computing platforms.

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve This paper aims to address the security, privacy, and safety issues faced by large language model (LLM) platforms (such as ChatGPT) when supporting third-party plugins. Specifically, the paper proposes a systematic evaluation framework to help LLM platform designers analyze and improve the security, privacy, and safety of current and future third-party integrated LLM platforms. #### Main Issues Include: 1. **Trust Issues with Third-Party Plugins**: - Third-party plugins are developed by unfamiliar developers and cannot be implicitly trusted. - Plugins may exploit natural language interfaces for ambiguous or imprecise interpretations, leading to security issues. 2. **Existing Security and Privacy Issues**: - Previous research indicates that third-party integrations on other computing platforms often lead to security and privacy issues. - For example, plugins may initiate prompt injection attacks or even take over the LLM platform. 3. **Limitations and Insufficient Review by Platform Vendors**: - Currently, some LLM platform vendors (such as OpenAI) impose few restrictions on third-party plugins, and the review process is relatively weak. - This may result in plugins going online without thorough review, increasing risks for users and the platform. 4. **Future Challenges**: - As LLM platform functionalities expand and third-party integrations increase, new security and privacy issues may arise in the future. - The paper proposes a systematic evaluation framework to provide a foundation for future research and practice to address these challenges. ### Main Contributions of the Framework 1. **Systematic Evaluation Framework**: - Developed a systematic evaluation framework centered around an attack taxonomy to assess the security, privacy, and safety of LLM platforms. 2. **Practical Application**: - Applied the framework in OpenAI's plugin ecosystem, identifying multiple plugins with potential attack risks. 3. **Guidance for Future Research and Practice**: - Reflected on the framework and discovered attacks, providing challenges and lessons for future researchers and industry practitioners to better protect the security, privacy, and safety of LLM platforms. ### Summary Through a systematic approach, the paper reveals various security, privacy, and safety issues faced by LLM platforms when supporting third-party plugins and proposes an evaluation framework to help designers and researchers address these challenges. This not only aids current LLM platforms but also provides important references for future LLM ecosystems.

LLM Platform Security: Applying a Systematic Evaluation Framework to OpenAI's ChatGPT Plugins

A New Era in LLM Security: Exploring Security Concerns in Real-World LLM-based Systems

Attacks on Third-Party APIs of Large Language Models

Exploring ChatGPT App Ecosystem: Distribution, Deployment and Security

Beyond the Safeguards: Exploring the Security Risks of ChatGPT

A survey on Large Language Model (LLM) security and privacy: The Good, The Bad, and The Ugly

SecGPT: An Execution Isolation Architecture for LLM-Based Systems

The Ethics of Interaction: Mitigating Security Threats in LLMs

RatGPT: Turning online LLMs into Proxies for Malware Attacks

Exploring the Limits of ChatGPT in Software Security Applications

Can We Trust Large Language Models Generated Code? A Framework for In-Context Learning, Security Patterns, and Code Evaluations Across Diverse LLMs

Threat Modelling and Risk Analysis for Large Language Model (LLM)-Powered Applications

Multi-step Jailbreaking Privacy Attacks on ChatGPT

Opening A Pandora's Box: Things You Should Know in the Era of Custom GPTs

Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection

Emerging Security Challenges of Large Language Models

A Qualitative Study on Using ChatGPT for Software Security: Perception vs. Practicality

Data Exposure from LLM Apps: An In-depth Investigation of OpenAI's GPTs

Large language models in 6G security: challenges and opportunities