Abstract:Generating realistic time series samples is crucial for stress-testing models and protecting user privacy by using synthetic data. In engineering and safety-critical applications, these samples must meet certain hard constraints that are domain-specific or naturally imposed by physics or nature. Consider, for example, generating electricity demand patterns with constraints on peak demand times. This can be used to stress-test the functioning of power grids during adverse weather conditions. Existing approaches for generating constrained time series are either not scalable or degrade sample quality. To address these challenges, we introduce Constrained Posterior Sampling (CPS), a diffusion-based sampling algorithm that aims to project the posterior mean estimate into the constraint set after each denoising update. Notably, CPS scales to a large number of constraints (~100) without requiring additional training. We provide theoretical justifications highlighting the impact of our projection step on sampling. Empirically, CPS outperforms state-of-the-art methods in sample quality and similarity to real time series by around 10% and 42%, respectively, on real-world stocks, traffic, and air quality datasets.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to generate real - time series samples that meet strict constraints. Specifically, the authors point out that in engineering and critical - safety applications, these time - series samples must satisfy certain hard constraints imposed by domain - specific or natural physical laws. For example, when generating electricity - demand patterns, it is necessary to constrain the peak - demand time for testing the functionality of the power grid under severe weather conditions. Existing methods for generating constrained time - series are either not scalable or will reduce the sample quality. To solve these problems, the authors introduce a sampling algorithm based on the diffusion model - **Constrained Posterior Sampling (CPS)**. This algorithm aims to project the posterior - mean estimate into the constraint set after each denoising update. CPS can handle a large number of constraints (about 100) without the need for additional training, and on real - world stock, traffic and air - quality data sets, CPS improves the sample quality and the similarity to the real - time series by about 10% and 42% respectively. ### Specific Problem Description 1. **Generate time - series samples that meet constraints**: The generated time - series must strictly adhere to various domain - specific constraints. For example, when generating daily open, high, low and close (OHLC) charts, the generated time - series samples should ensure that the open and close prices are between the high and low prices. 2. **Maintain sample quality**: While imposing constraints, ensure that the quality of the generated time - series samples is not affected, that is, the generated samples can still reflect the situation in the real world. 3. **Scalability and flexibility**: Existing methods are often not scalable or require retraining the model when dealing with a large number of constraints. CPS makes the algorithm able to handle a large number of constraints without sacrificing sample quality by introducing a projection step. ### Solution Overview The CPS method proposed by the authors solves the above problems in the following ways: - **Projection step**: After each denoising update, project the posterior - mean estimate into the constraint set, thereby ensuring that the generated samples meet all constraint conditions. - **No additional training required**: CPS can handle a large number of constraint conditions without the need for additional training, which makes it more flexible and efficient in practical applications. - **Theoretical analysis and experimental verification**: The authors provide a detailed theoretical analysis to prove the impact of the projection step on sampling, and verify the superior performance of CPS on multiple real - world data sets through experiments. In conclusion, this paper solves the challenge of generating time - series samples that meet strict constraint conditions while maintaining the quality and diversity of the samples by proposing the CPS method.

Constrained Posterior Sampling: Time Series Generation with Hard Constraints

On the Constrained Time-Series Generation Problem

Multi-Step Denoising Scheduled Sampling: Towards Alleviating Exposure Bias for Diffusion Models

Adaptive Sampling for Deep Learning via Efficient Nonparametric Proxies

A Survey of Estimation Methods for Sparse High-dimensional Time Series Models

Constrained Sampling with Primal-Dual Langevin Monte Carlo

Smart Sampling Algorithm for Surrogate Model Development

Fast Sampling for Time-Varying Determinantal Point Processes

Learning Rate Free Sampling in Constrained Domains

Accelerating Diffusion Sampling with Optimized Time Steps

Divide-and-Conquer Posterior Sampling for Denoising Diffusion Priors

An autocorrelated conditioned Latin hypercube method for temporal or spatial sampling and predictions

Posterior sampling via Langevin dynamics based on generative priors

Constrained inference through posterior projections

Optimal Sampling Designs for Multi-dimensional Streaming Time Series with Application to Power Grid Sensor Data

Confidence partitioning sampling filtering

Intelligent sampling for surrogate modeling, hyperparameter optimization, and data analysis

Compressed Least Squares Algorithm of Continuous-Time Linear Stochastic Regression Model Using Sampling Data

Sampling approaches to reduce very frequent seasonal time series

Surrogate time series