Abstract:Background Synthetic data have been proposed as a solution for sharing anonymized versions of sensitive biomedical datasets. Ideally, synthetic data should preserve the structure and statistical properties of the original data, while protecting the privacy of the individual subjects. Differential Privacy (DP) is currently considered the gold standard approach for balancing this trade-off. Objectives The aim of this study is to investigate how trustworthy are group differences discovered by independent sample tests from DP-synthetic data. The evaluation is carried out in terms of the tests' Type I and Type II errors. With the former, we can quantify the tests' validity, i.e., whether the probability of false discoveries is indeed below the significance level, and the latter indicates the tests' power in making real discoveries. Methods We evaluate the Mann–Whitney U test, Student's t -test, chi-squared test, and median test on DP-synthetic data. The private synthetic datasets are generated from real-world data, including a prostate cancer dataset ( n = 500) and a cardiovascular dataset ( n = 70,000), as well as on bivariate and multivariate simulated data. Five different DP-synthetic data generation methods are evaluated, including two basic DP histogram release methods and MWEM, Private-PGM, and DP GAN algorithms. Conclusion A large portion of the evaluation results expressed dramatically inflated Type I errors, especially at levels of ε ≤ 1. This result calls for caution when releasing and analyzing DP-synthetic data: low p -values may be obtained in statistical tests simply as a byproduct of the noise added to protect privacy. A DP Smoothed Histogram-based synthetic data generation method was shown to produce valid Type I error for all privacy levels tested but required a large original dataset size and a modest privacy budget ( ε ≥ 5) in order to have reasonable Type II error levels. Received: 31 May 2023 Accepted: 25 July 2024 Accepted Manuscript online: 13 August 2024 Article published online: 09 September 2024 © 2024. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution License, permitting unrestricted use, distribution, and reproduction so long as the original work is properly cited. (https://creativecommons.org/licenses/by/4.0/) Georg Thieme Verlag KG Rüdigerstraße 14, 70469 Stuttgart, Germany

Collaborative learning from distributed data with differentially private synthetic data

Collaborative Learning From Distributed Data With Differentially Private Synthetic Twin Data

PrivSyn: Differentially Private Data Synthesis

Generating Poisson-Distributed Differentially Private Synthetic Data

Differentially Private Synthetic Data with Private Density Estimation

Differentially Private Synthetic Data: Applied Evaluations and Enhancements

Comparative Study of Differentially Private Data Synthesis Methods

End to End Collaborative Synthetic Data Generation

Evaluating Differentially Private Synthetic Data Generation in High-Stakes Domains

Secure and Differentially Private Bayesian Learning on Distributed Data

Privacy Preserving Analytics on Distributed Medical Data

Does Differentially Private Synthetic Data Lead to Synthetic Discoveries?

Differential Privacy-enabled Federated Learning for Sensitive Health Data

A Data Synthesis Approach Based on Local Differential Privacy

Federated learning for preserving data privacy in collaborative healthcare research

Tabular Data Synthesis with Differential Privacy: A Survey

Synthetic data for privacy-preserving clinical risk prediction

pMSE Mechanism: Differentially Private Synthetic Data with Maximal Distributional Similarity

Inference With Combining Rules From Multiple Differentially Private Synthetic Datasets

Differentially Private Low-dimensional Synthetic Data from High-dimensional Datasets