Abstract:Large language models (LLMs), like ChatGPT, have greatly simplified text generation tasks. However, they have also raised concerns about privacy risks such as data leakage and unauthorized data collection. Existing solutions for privacy-preserving inference face practical challenges related to computation time and communication costs. In this paper, we propose InferDPT, the first practical framework for the privacy-preserving Inference of black-box LLMs, implementing Differential Privacy in Text generation. InferDPT comprises two key modules: the "perturbation module" utilizes the exponential mechanism to generate a perturbed prompt, facilitating privacy-preserving inference with black-box LLMs, and the "extraction module", inspired by knowledge distillation and retrieval-augmented generation, extracts coherent and consistent text from the perturbed generation result, ensuring successful text generation completion. To address privacy concerns related to previous exponential mechanisms' susceptibility to embedding revision attacks, we introduce RANTEXT, a novel differential privacy mechanism integrated into the perturbation module of InferDPT, which introduces the concept of "RANdom adjacency" for TEXT perturbation within the prompt. Experimental results across three datasets demonstrate that the text generation quality of InferDPT is comparable to that of non-private GPT-4, and RANTEXT surpasses existing state-of-the-art mechanisms, namely, SANTEXT+ and CUSTEXT+ in the trade-off between privacy and utility. Even with an privacy parameter epsilon value of 6.0, RANTEXT achieves an average privacy protection rate exceeding 90% against embedding revision attacks, which is 0.58 times higher than that of SANTEXT+ and 3.35 times higher than that of CUSTEXT+.

What problem does this paper attempt to address?

### Problems the Paper Aims to Solve The paper aims to address the issue of privacy protection in text generation tasks using large language models (LLMs). Specifically, it focuses on how to protect the privacy of user-uploaded original documents during inference with black-box LLMs, preventing data leakage and unauthorized information collection. ### Background and Challenges 1. **Development of Large Language Models**: - In recent years, large language models (LLMs) represented by ChatGPT have made significant progress in text generation tasks, greatly simplifying the work of text generation. - However, these models have also raised concerns about privacy risks, such as data leakage and unauthorized information collection. 2. **Limitations of Existing Solutions**: - **Computation Time and Communication Costs**: Existing privacy-preserving inference methods face challenges in terms of computation time and communication costs in practical applications. - **Applicability to Classification Tasks**: Some methods (such as SANTEXT+ and CUSTEXT+) are mainly suitable for classification tasks and perform poorly in text generation tasks. - **Vulnerability to Embedding Inversion Attacks**: Existing differential privacy mechanisms are vulnerable to embedding inversion attacks, leading to privacy leakage. ### Proposed Solution 1. **InferDPT Framework**: - **Perturbation Module**: Utilizes differential privacy mechanisms to generate perturbed prompts, protecting the privacy of user-uploaded original documents. - **Extraction Module**: Extracts coherent text from the perturbed generation results and reconstructs the output through a local language model, ensuring alignment with the original prompt. 2. **RANTEXT Mechanism**: - **Random Adjacency List**: Introduces the concept of a random adjacency list, perturbing each word or token to enhance defense against embedding inversion attacks. - **Laplace Distribution**: Dynamically determines the size of the random adjacency list and samples new words or tokens from it to replace the original private words or tokens. ### Experimental Results - **Generation Quality**: Experimental results show that the generation quality of InferDPT is comparable to that of non-privacy-preserving GPT-4. - **Privacy Protection Rate**: Even with a privacy parameter ε value of 6.0, RANTEXT can achieve an average privacy protection rate of over 90%, significantly outperforming existing SANTEXT+ and CUSTEXT+ mechanisms. ### Summary The paper proposes a framework named InferDPT for achieving privacy-preserving inference in black-box large language models. By introducing the RANTEXT mechanism, this framework effectively addresses the vulnerabilities of existing differential privacy mechanisms in text generation tasks, providing higher privacy protection and generation quality.

InferDPT: Privacy-Preserving Inference for Black-box Large Language Model

Differentially Private Synthetic Data via Foundation Model APIs 2: Text

SecureGPT: A Framework for Multi-Party Privacy-Preserving Transformer Inference in GPT

Private prediction for large-scale synthetic text generation

Textual Differential Privacy for Context-Aware Reasoning with Large Language Model

Reconstruction of Differentially Private Text Sanitization via Large Language Models

Locally Differentially Private Document Generation Using Zero Shot Prompting

Privacy-Preserving Retrieval Augmented Generation with Differential Privacy

Privacy-Preserving In-Context Learning for Large Language Models

PrivacyRestore: Privacy-Preserving Inference in Large Language Models via Privacy Removal and Restoration

Privacy Preserving Large Language Models: ChatGPT Case Study Based Vision and Framework

LMO-DP: Optimizing the Randomization Mechanism for Differentially Private Fine-Tuning (Large) Language Models

PDSS: A Privacy-Preserving Framework for Step-by-Step Distillation of Large Language Models

Differentially Private Language Models for Secure Data Sharing

ConfusionPrompt: Practical Private Inference for Online Large Language Models

Inference Privacy: Properties and Mechanisms

Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipe

IDT: Dual-Task Adversarial Attacks for Privacy Protection

Differentially Private Next-Token Prediction of Large Language Models

Split-and-Denoise: Protect large language model inference with local differential privacy