Abstract:Building on the remarkable achievements in generative sampling of natural images, we propose an innovative challenge, potentially overly ambitious, which involves generating samples of entire multivariate time series that resemble images. However, the statistical challenge lies in the small sample size, sometimes consisting of a few hundred subjects. This issue is especially problematic for deep generative models that follow the conventional approach of generating samples from a canonical distribution and then decoding or denoising them to match the true data distribution. In contrast, our method is grounded in information theory and aims to implicitly characterize the distribution of images, particularly the (global and local) dependency structure between pixels. We achieve this by empirically estimating its KL-divergence in the dual form with respect to the respective marginal distribution. This enables us to perform generative sampling directly in the optimized 1-D dual divergence space. Specifically, in the dual space, training samples representing the data distribution are embedded in the form of various clusters between two end points. In theory, any sample embedded between those two end points is in-distribution w.r.t. the data distribution. Our key idea for generating novel samples of images is to interpolate between the clusters via a walk as per gradients of the dual function w.r.t. the data dimensions. In addition to the data efficiency gained from direct sampling, we propose an algorithm that offers a significant reduction in sample complexity for estimating the divergence of the data distribution with respect to the marginal distribution. We provide strong theoretical guarantees along with an extensive empirical evaluation using many real-world datasets from diverse domains, establishing the superiority of our approach w.r.t. state-of-the-art deep learning methods.

f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization

Exploring Guided Sampling of Conditional GANs

Training Neural Samplers with Reverse Diffusive KL Divergence

A Convex Duality Framework for GANs

Sampling Generative Networks

Learning to Draw Samples: With Application to Amortized MLE for Generative Adversarial Learning

GANs Settle Scores!

Sketch-then-Edit Generative Adversarial Network

A New Distributed Method for Training Generative Adversarial Networks

A Bayesian Non-parametric Approach to Generative Models: Integrating Variational Autoencoder and Generative Adversarial Networks using Wasserstein and Maximum Mean Discrepancy

Deep Generative Sampling in the Dual Divergence Space: A Data-efficient & Interpretative Approach for Generative AI

Adversarial Bayesian Simulation

On Divergence Measures for Training GFlowNets

Utilizing Amari-Alpha Divergence to Stabilize the Training of Generative Adversarial Networks

Bridging the Gap Between $f$-GANs and Wasserstein GANs

Calibrated adversarial algorithms for generative modelling

Local Convergence of Gradient Descent-Ascent for Training Generative Adversarial Networks

Variational Approaches for Auto-Encoding Generative Adversarial Networks

The residual generator: An improved divergence minimization framework for GAN

Stabilizing Training of Generative Adversarial Nets via Langevin Stein Variational Gradient Descent

Fictitious GAN: Training GANs with Historical Models