Abstract:The availability of textual data depicting human-centered features and behaviors is crucial for many data mining and machine learning tasks. However, data containing personal information should be anonymized prior making them available for secondary use. A variety of text anonymization methods have been proposed in the last years, which are standardly evaluated by comparing their outputs with human-based anonymizations. The residual disclosure risk is estimated with the recall metric, which quantifies the proportion of manually annotated re-identifying terms successfully detected by the anonymization algorithm. Nevertheless, recall is not a risk metric, which leads to several drawbacks. First, it requires a unique ground truth, and this does not hold for text anonymization, where several masking choices could be equally valid to prevent re-identification. Second, it relies on human judgements, which are inherently subjective and prone to errors. Finally, the recall metric weights terms uniformly, thereby ignoring the fact that the influence on the disclosure risk of some missed terms may be much larger than of others. To overcome these drawbacks, in this paper we propose a novel method to evaluate the disclosure risk of anonymized texts by means of an automated re-identification attack. We formalize the attack as a multi-class classification task and leverage state-of-the-art neural language models to aggregate the data sources that attackers may use to build the classifier. We illustrate the effectiveness of our method by assessing the disclosure risk of several methods for text anonymization under different attack configurations. Empirical results show substantial privacy risks for most existing anonymization methods.

Towards Quantifying The Privacy Of Redacted Text

Privacy Guarantees for De-identifying Text Transformations

Evaluating the Efficacy of AI Techniques in Textual Anonymization: A Comparative Study

Benchmarking Advanced Text Anonymisation Methods: A Comparative Study on Novel and Traditional Approaches

IncogniText: Privacy-enhancing Conditional Text Anonymization via LLM-based Private Attribute Randomization

Keep It Private: Unsupervised Privatization of Online Text

SynTF: Synthetic and Differentially Private Term Frequency Vectors for Privacy-Preserving Text Mining

How reparametrization trick broke differentially-private text representation learning

TextObfuscator: Making Pre-trained Language Model a Privacy Protector via Obfuscating Word Representations

Text Revealer: Private Text Reconstruction via Model Inversion Attacks against Transformers

To show or not to show: Redacting sensitive text from videos of electronic displays

ADePT: Auto-encoder based Differentially Private Text Transformation

Evaluating the disclosure risk of anonymized documents via a machine learning-based re-identification attack

Textwash -- automated open-source text anonymisation

NAP^2: A Benchmark for Naturalness and Privacy-Preserving Text Rewriting by Learning from Human

Crowdsourcing on Sensitive Data with Privacy-Preserving Text Rewriting

Neural Text Sanitization with Explicit Measures of Privacy Risk

An Easy-to-use and Robust Approach for the Differentially Private De-Identification of Clinical Textual Documents

Neural Text Sanitization with Privacy Risk Indicators: An Empirical Analysis

Guiding Text-to-Text Privatization by Syntax

Just Rewrite It Again: A Post-Processing Method for Enhanced Semantic Similarity and Privacy Preservation of Differentially Private Rewritten Text