Differentially Private Language Models for Secure Data Sharing

Justus Mattern,Zhijing Jin,Benjamin Weggenmann,Bernhard Schoelkopf,Mrinmaya Sachan

DOI: https://doi.org/10.48550/arXiv.2210.13918

2022-10-26

Abstract:To protect the privacy of individuals whose data is being shared, it is of high importance to develop methods allowing researchers and companies to release textual data while providing formal privacy guarantees to its originators. In the field of NLP, substantial efforts have been directed at building mechanisms following the framework of local differential privacy, thereby anonymizing individual text samples before releasing them. In practice, these approaches are often dissatisfying in terms of the quality of their output language due to the strong noise required for local differential privacy. In this paper, we approach the problem at hand using global differential privacy, particularly by training a generative language model in a differentially private manner and consequently sampling data from it. Using natural language prompts and a new prompt-mismatch loss, we are able to create highly accurate and fluent textual datasets taking on specific desired attributes such as sentiment or topic and resembling statistical properties of the training data. We perform thorough experiments indicating that our synthetic datasets do not leak information from our original data and are of high language quality and highly suitable for training models for further analysis on real-world data. Notably, we also demonstrate that training classifiers on private synthetic data outperforms directly training classifiers on real data with DP-SGD.

Machine Learning,Computation and Language,Cryptography and Security

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to protect personal privacy when sharing text data. Specifically, researchers are concerned with how to enable researchers and companies to publish text data without revealing the personal information of the original data providers. Although the traditional local differential privacy method can anonymize individual text samples, in practical applications, a large amount of noise needs to be added to meet the requirements of local differential privacy, resulting in poor - quality generated text and difficulty in retaining the semantic information of the original text.

For this reason, this paper proposes a method based on global differential privacy, which generates a synthetic text data set with privacy protection by training a generative language model. This method can not only generate high - quality text, but also maintain statistical characteristics similar to the training data, so it is suitable for further analysis tasks. In addition, the research also shows that the performance of the classifier trained on private synthetic data is better than that of the classifier trained directly on the real data processed by differential privacy stochastic gradient descent (DP - SGD).

The main contribution of the paper is to provide an effective method that ensures both the privacy of data and the quality of data, which is of great significance for scenarios where sensitive data needs to be shared, such as external consultants conducting data analysis or providing high - quality benchmark data sets for the research community.