Abstract:The diffusion model has shown remarkable performance in modeling data distributions and synthesizing data. However, the vanilla diffusion model requires complete or fully observed data for training. Incomplete data is a common issue in various real-world applications, including healthcare and finance, particularly when dealing with tabular datasets. This work presents a unified and principled diffusion-based framework for learning from data with missing values under various missing mechanisms. We first observe that the widely adopted "impute-then-generate" pipeline may lead to a biased learning objective. Then we propose to mask the regression loss of Denoising Score Matching in the training phase. We prove the proposed method is consistent in learning the score of data distributions, and the proposed training objective serves as an upper bound for the negative likelihood in certain cases. The proposed framework is evaluated on multiple tabular datasets using realistic and efficacious metrics and is demonstrated to outperform state-of-the-art diffusion model on tabular data with "impute-then-generate" pipeline by a large margin.

What problem does this paper attempt to address?

### The problems the paper attempts to solve The paper "MissDiff: Training Diffusion Models on Tabular Data with Missing Values" aims to solve the problem of training diffusion models in the presence of missing values. Specifically, the paper focuses on how to train a generative model on tabular data with a large number of missing values and generate complete data. This problem is very common in practical applications, especially in fields such as healthcare and finance, where data missing is a common and difficult problem. ### Background and motivation Diffusion models perform well in modeling data distributions and synthesizing data, but traditional diffusion models require complete or fully - observed data for training. However, in real - world applications, data missing is a common problem, especially when dealing with tabular data. For example, a patient's respiration rate may be missing because it is considered unnecessary or accidentally not recorded. In addition, some information may be difficult or even dangerous to obtain, such as information obtained through biopsy. ### Solutions The paper proposes a framework based on diffusion models, called **MissDiff**, for learning from data with missing values. The main contributions include: 1. **Proposing a new training method**: By masking the regression loss of denoising score matching (Denoising Score Matching) during the training process, the bias caused by the traditional "impute - then - generate" method is avoided. 2. **Theoretical guarantee**: It is proved that the proposed method is consistent in recovering the score function of the data distribution, and in some cases, the proposed training objective can be an upper bound of the negative log - likelihood. 3. **Experimental verification**: Extensive numerical experiments have been carried out on multiple real - world tabular datasets, and the results show that MissDiff is significantly superior to existing methods in dealing with tabular data with missing values. ### Method overview 1. **Data representation**: For each data point \( \mathbf{x} \), a binary mask \( \mathbf{m} \) is used to represent which features are missing. The observed data \( \mathbf{x}_{\text{obs}} \) can be represented as: \[ \mathbf{x}_{\text{obs}} = \mathbf{x} \odot \mathbf{m} + \mathbf{na} \odot (1 - \mathbf{m}) \] where \( \odot \) represents element - wise multiplication and \( \mathbf{na} \) represents missing values. 2. **Denoising score matching**: A new denoising score matching method is proposed, which trains the model by masking the loss at the missing values: \[ \theta^* = \arg \min_{\theta} J_{\text{DSM}}(\theta) := \frac{T}{2} \mathbb{E}_t \left[ \lambda(t) \mathbb{E}_{\mathbf{x}_{\text{obs}}(0)} \mathbb{E}_{\mathbf{x}_{\text{obs}}(t) | \mathbf{x}_{\text{obs}}(0)} \left\| \left( s_{\theta}(\mathbf{x}_{\text{obs}}(t), t) - \nabla_{\mathbf{x}_{\text{obs}}(t)} \log p(\mathbf{x}_{\text{obs}}(t) | \mathbf{x}_{\text{obs}}(0)) \right) \odot \mathbf{m} \right\|_2^2 \right] \] 3. **Theoretical analysis**: It is proved that under certain assumptions, the proposed denoising score matching method can learn the complete data score function and

MissDiff: Training Diffusion Models on Tabular Data with Missing Values

Denoising Diffusion Straightforward Models for Energy Conversion Monitoring Data Imputation

DiffImpute: Tabular Data Imputation With Denoising Diffusion Probabilistic Model

Diffusion Models for Tabular Data Imputation and Synthetic Data Generation

TabDiff: a Multi-Modal Diffusion Model for Tabular Data Generation

Self-Supervision Improves Diffusion Models for Tabular Data Imputation

An Observed Value Consistent Diffusion Model for Imputing Missing Values in Multivariate Time Series

TabDDPM: Modelling Tabular Data with Diffusion Models

Rethinking the Diffusion Models for Numerical Tabular Data Imputation from the Perspective of Wasserstein Gradient Flow

Unleashing the Potential of Diffusion Models for Incomplete Data Imputation

A Training-Free Conditional Diffusion Model for Learning Stochastic Dynamical Systems

Data Augmentation via Diffusion Model to Enhance AI Fairness

On conditional diffusion models for PDE simulations

Training Class-Imbalanced Diffusion Model Via Overlap Optimization

Balanced Mixed-Type Tabular Data Synthesis with Diffusion Models

Unmasking Bias in Diffusion Model Training

TabADM: Unsupervised Tabular Anomaly Detection with Diffusion Models

Addressing Negative Transfer in Diffusion Models

Latent Space Score-based Diffusion Model for Probabilistic Multivariate Time Series Imputation

FedTabDiff: Federated Learning of Diffusion Probabilistic Models for Synthetic Mixed-Type Tabular Data Generation

Tabular Data Generation using Binary Diffusion