Constrained Posterior Sampling: Time Series Generation with Hard Constraints

Sai Shankar Narasimhan,Shubhankar Agarwal,Litu Rout,Sanjay Shakkottai,Sandeep P. Chinchali
2024-10-16
Abstract:Generating realistic time series samples is crucial for stress-testing models and protecting user privacy by using synthetic data. In engineering and safety-critical applications, these samples must meet certain hard constraints that are domain-specific or naturally imposed by physics or nature. Consider, for example, generating electricity demand patterns with constraints on peak demand times. This can be used to stress-test the functioning of power grids during adverse weather conditions. Existing approaches for generating constrained time series are either not scalable or degrade sample quality. To address these challenges, we introduce Constrained Posterior Sampling (CPS), a diffusion-based sampling algorithm that aims to project the posterior mean estimate into the constraint set after each denoising update. Notably, CPS scales to a large number of constraints (~100) without requiring additional training. We provide theoretical justifications highlighting the impact of our projection step on sampling. Empirically, CPS outperforms state-of-the-art methods in sample quality and similarity to real time series by around 10% and 42%, respectively, on real-world stocks, traffic, and air quality datasets.
Machine Learning,Artificial Intelligence,Signal Processing
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to generate real - time series samples that meet strict constraints. Specifically, the authors point out that in engineering and critical - safety applications, these time - series samples must satisfy certain hard constraints imposed by domain - specific or natural physical laws. For example, when generating electricity - demand patterns, it is necessary to constrain the peak - demand time for testing the functionality of the power grid under severe weather conditions. Existing methods for generating constrained time - series are either not scalable or will reduce the sample quality. To solve these problems, the authors introduce a sampling algorithm based on the diffusion model - **Constrained Posterior Sampling (CPS)**. This algorithm aims to project the posterior - mean estimate into the constraint set after each denoising update. CPS can handle a large number of constraints (about 100) without the need for additional training, and on real - world stock, traffic and air - quality data sets, CPS improves the sample quality and the similarity to the real - time series by about 10% and 42% respectively. ### Specific Problem Description 1. **Generate time - series samples that meet constraints**: The generated time - series must strictly adhere to various domain - specific constraints. For example, when generating daily open, high, low and close (OHLC) charts, the generated time - series samples should ensure that the open and close prices are between the high and low prices. 2. **Maintain sample quality**: While imposing constraints, ensure that the quality of the generated time - series samples is not affected, that is, the generated samples can still reflect the situation in the real world. 3. **Scalability and flexibility**: Existing methods are often not scalable or require retraining the model when dealing with a large number of constraints. CPS makes the algorithm able to handle a large number of constraints without sacrificing sample quality by introducing a projection step. ### Solution Overview The CPS method proposed by the authors solves the above problems in the following ways: - **Projection step**: After each denoising update, project the posterior - mean estimate into the constraint set, thereby ensuring that the generated samples meet all constraint conditions. - **No additional training required**: CPS can handle a large number of constraint conditions without the need for additional training, which makes it more flexible and efficient in practical applications. - **Theoretical analysis and experimental verification**: The authors provide a detailed theoretical analysis to prove the impact of the projection step on sampling, and verify the superior performance of CPS on multiple real - world data sets through experiments. In conclusion, this paper solves the challenge of generating time - series samples that meet strict constraint conditions while maintaining the quality and diversity of the samples by proposing the CPS method.