Abstract:BACKGROUND: Ensuring safe adoption of AI tools in healthcare hinges on access to sufficient data for training, testing and validation. Synthetic data has been suggested in response to privacy concerns and regulatory requirements and can be created by training a generator on real data to produce a dataset with similar statistical properties. Competing metrics with differing taxonomies for quality evaluation have been proposed, resulting in a complex landscape. Optimising quality entails balancing considerations that make the data fit for use, yet relevant dimensions are left out of existing frameworks.METHOD: We performed a comprehensive literature review on the use of quality evaluation metrics on synthetic data within the scope of synthetic tabular healthcare data using deep generative methods. Based on this and the collective team experiences, we developed a conceptual framework for quality assurance. The applicability was benchmarked against a practical case from the Dutch National Cancer Registry.CONCLUSION: We present a conceptual framework for quality assuranceof synthetic data for AI applications in healthcare that aligns diverging taxonomies, expands on common quality dimensions to include the dimensions of Fairness and Carbon footprint, and proposes stages necessary to support real-life applications. Building trust in synthetic data by increasing transparency and reducing the safety risk will accelerate the development and uptake of trustworthy AI tools for the benefit of patients.DISCUSSION: Despite the growing emphasis on algorithmic fairness and carbon footprint, these metrics were scarce in the literature review. The overwhelming focus was on statistical similarity using distance metrics while sequential logic detection was scarce. A consensus-backed framework that includes all relevant quality dimensions can provide assurance for safe and responsible real-life applications of synthetic data. As the choice of appropriate metrics are highly context dependent, further research is needed on validation studies to guide metric choices and support the development of technical standards.

NFDI4Health workflow and service for synthetic data generation, assessment and risk management

Privacy Risk Assessment for Synthetic Longitudinal Health Data

Synthetic data generation: State of the art in health care domain

Interoperable synthetic health data with SyntHIR to enable the development of CDSS tools

On the Trade-Off between Fidelity, Utility and Privacy of Synthetic Patient Data

Synthetic Data in Healthcare

Enabling Synthetic Data adoption in regulated domains

Synthetic datasets for open software development in rare disease research

A primer on synthetic health data

Synthetic data generation for a longitudinal cohort study – evaluation, method extension and reproduction of published data analysis results

Generating Synthetic Data for Medical Imaging

INSAFEDARE Project: Innovative Applications of Assessment and Assurance of Data and Synthetic Data for Regulatory Decision Support

Can I trust my fake data – A comprehensive quality assessment framework for synthetic tabular data in healthcare

Synthetic data generation methods in healthcare: A review on open-source tools and methods

Generating high-fidelity synthetic patient data for assessing machine learning healthcare software

Synthetic Data Generator for Adaptive Interventions in Global Health

Fidelity and Privacy of Synthetic Medical Data

Synthetic data generation for a longitudinal cohort study -- Evaluation, method extension and reproduction of published data analysis results

Synthetic data: How could it be used for infectious disease research?

Patient-centric synthetic data generation, no reason to risk re-identification in biomedical data analysis

A Novel Taxonomy for Navigating and Classifying Synthetic Data in Healthcare Applications