Abstract:Synthetic data is often presented as a method for sharing sensitive information in a privacy-preserving manner by reproducing the global statistical properties of the original data without disclosing sensitive information about any individual. In practice, as with other anonymization methods, privacy risks cannot be entirely eliminated. The residual privacy risks need instead to be ex-post assessed. We present Anonymeter, a statistical framework to jointly quantify different types of privacy risks in synthetic tabular datasets. We equip this framework with attack-based evaluations for the singling out, linkability, and inference risks, the three key indicators of factual anonymization according to the European General Data Protection Regulation (GDPR). To the best of our knowledge, we are the first to introduce a coherent and legally aligned evaluation of these three privacy risks for synthetic data, and to design privacy attacks which model directly the singling out and linkability risks. We demonstrate the effectiveness of our methods by conducting an extensive set of experiments that measure the privacy risks of data with deliberately inserted privacy leakages, and of synthetic data generated with and without differential privacy. Our results highlight that the three privacy risks reported by our framework scale linearly with the amount of privacy leakage in the data. Furthermore, we observe that synthetic data exhibits the lowest vulnerability against linkability, indicating one-to-one relationships between real and synthetic data records are not preserved. Finally, we demonstrate quantitatively that Anonymeter outperforms existing synthetic data privacy evaluation frameworks both in terms of detecting privacy leaks, as well as computation speed. To contribute to a privacy-conscious usage of synthetic data, we open source Anonymeter at <a class="link-external link-https" href="https://github.com/statice/anonymeter" rel="external noopener nofollow">this https URL</a>.

FCA-based reasoning for privacy

A Decision Tree Approach for Assessing and Mitigating Background and Identity Disclosure Risks.

Managing Your Private and Public Data: Bringing Down Inference Attacks Against Your Privacy

Privacy Risk Assessment: From Art to Science, By Metrics

Privacy-Preserving Data Analysis for the Federal Statistical Agencies

Increasing Transparent and Accountable Use of Data by Quantifying the Actual Privacy Risk in Interactive Record Linkage

A Novel Microdata Privacy Disclosure Risk Measure

Quantifying Privacy: A Novel Entropy-Based Measure of Disclosure Risk

Protecting privacy in the age of big data: exploring data linking methods for quasi-identifier selection

A Model-oriented Reasoning Framework for Privacy Analysis of Complex Systems

Anonymization: The imperfect science of using data while preserving privacy

A Cautionary Tale: On the Role of Reference Data in Empirical Privacy Defenses

Encode, Shuffle, Analyze Privacy Revisited: Formalizations and Empirical Evaluation.

Privacy and Statistical Risk: Formalisms and Minimax Bounds

On some consequences of the permutation paradigm for data anonymization: centrality of permutation matrices, universal measures of disclosure risk and information loss, evaluation by dominance

InvMetrics: Measuring Privacy Risks for Split Model–Based Customer Behavior Analysis

Privacy Risk Assessment for Synthetic Longitudinal Health Data

A Unified Framework for Quantifying Privacy Risk in Synthetic Data

How to Quantify Graph De-anonymization Risks.

Data Fusion Challenges Privacy: What Can Privacy Regulation Do?

Privacy risk from synthetic data: practical proposals