Abstract:The ever-increasing use of generative models in various fields where tabular data is used highlights the need for robust and standardized validation metrics to assess the similarity between real and synthetic data. Current methods lack a unified framework and rely on diverse and often inconclusive statistical measures. Divergences, which quantify discrepancies between data distributions, offer a promising avenue for validation. However, traditional approaches calculate divergences independently for each feature due to the complexity of joint distribution modeling. This paper addresses this challenge by proposing a novel approach that uses divergence estimation to overcome the limitations of marginal comparisons. Our core contribution lies in applying a divergence estimator to build a validation metric considering the joint distribution of real and synthetic data. We leverage a probabilistic classifier to approximate the density ratio between datasets, allowing the capture of complex relationships. We specifically calculate two divergences: the well-known Kullback-Leibler (KL) divergence and the Jensen-Shannon (JS) divergence. KL divergence offers an established use in the field, while JS divergence is symmetric and bounded, providing a reliable metric. The efficacy of this approach is demonstrated through a series of experiments with varying distribution complexities. The initial phase involves comparing estimated divergences with analytical solutions for simple distributions, setting a benchmark for accuracy. Finally, we validate our method on a real-world dataset and its corresponding synthetic counterpart, showcasing its effectiveness in practical applications. This research offers a significant contribution with applicability beyond tabular data and the potential to improve synthetic data validation in various fields.

Statistical Inference of Normal Distribution Based on Several Divergence Measures: A Comparative Study

Detecting abnormal situations using the Kullback-Leibler divergence.

The Logarithmic Super Divergence and its use in Statistical Inference

Bayesian influence diagnostics using normalizing functional Bregman divergence

Relative Density-Ratio Estimation for Robust Distribution Comparison

A Unifying Framework for Some Directed Distances in Statistics

Comparative evaluation of goodness of fit tests for normal distribution using simulation and empirical data

Statistical divergences in high-dimensional hypothesis testing and a modern technique for estimating them

On the Kullback-Leibler divergence between discrete normal distributions

Power analysis of several normality tests: A Monte Carlo simulation study

R-divergence for Estimating Model-oriented Distribution Discrepancy

A Ball Divergence Based Measure For Conditional Independence Testing

A discrete extension of the Burr-Hatke distribution: Generalized hypergeometric functions, different inference techniques, simulation ranking with modeling and analysis of sustainable count data

The Unit Folded Normal Distribution: A New Unit Probability Distribution with the Estimation Procedures, Quantile Regression Modeling and Educational Attainment Applications

A Neural Network-Based Approach to Normality Testing for Dependent Data

Computation of the distribution of model accuracy statistics in machine learning: Comparison between analytically derived distributions and simulation-based methods

Synthetic Tabular Data Validation: A Divergence-Based Approach

Statistical Detection Of Collective Data Fraud

Divergence Measure of Belief Function and Its Application in Data Fusion

Monte Carlo comparison of normality tests based on varentropy estimators

Two-Sample Inference in Highly Dispersed Negative Binomial Models