Abstract:Diffusion models have become a prevalent framework in deep generative modeling across various modalities. However, despite producing high quality results, these models are computationally expensive and suffer from slow convergence. In this work, we address these challenges in image generation by leveraging the wavelet domain, which decomposes images into low and high-frequency components, each at half the resolution of the original image in both height and width. We observe that prioritizing the learning of low-frequency components over high-frequency details and masking out unnecessary high-frequency content in wavelet space can significantly enhance training convergence and reduce computational demands. This strategy simplifies the complexity associated with high-frequency details during training, allowing the model to capture the most representative features of the data distribution while maintaining a balance in detail preservation. To facilitate controlled learning across different wavelet coefficients, we employ a multitask loss function, with each task corresponding to the learning of a distinct wavelet subband. Additionally, to ensure consistency among wavelet coefficients, which is crucial for accurate reconstruction in pixel space, we introduce a multispectral cross-attention mechanism to aid the joint generation of different wavelet coefficients. The sampling process involves jointly generating wavelet coefficients, followed by an inverse wavelet transform to convert them back to pixel space. Our approach not only improves the training efficiency for unconditional image generation compared with the standard denoising diffusion probabilistic model (vanilla DDPM) but also uniquely supports the generation of high-frequency content conditioned on a low-resolution image, enabling both image generation and upsampling within a single model. To our knowledge, this capability is novel. Our model demonstrates superior performance in image generation compared with baseline models on the STL-10 dataset, as evidenced by improved Frećhet inception distance (FID) and recall scores.

Diffusion-Based Bit-Depth Expansion

MultiSpectral diffusion: joint generation of wavelet coefficients for image synthesis and upsampling

Contour wavelet diffusion: A fast and high‐quality image generation model

Deep Reconstruction of Least Significant Bits for Bit-Depth Expansion

From 2D Extrapolation to 1D Interpolation: Content Adaptive Image Bit-Depth Expansion

Efficient image generation with Contour Wavelet Diffusion

Lossy Image Compression with Foundation Diffusion Models

Lighter but Efficient Bit-Depth Expansion Network

Diff-2-in-1: Bridging Generation and Dense Perception with Diffusion Models

High Frequency Matters: Uncertainty Guided Image Compression with Wavelet Diffusion

Diffusion Models Without Attention

MWRD (Mamba Wavelet Reverse Diffusion)—An Efficient Fundus Image Enhancement Network Based on an Improved State-Space Model

A Residual Diffusion Model for High Perceptual Quality Codec Augmentation

FAM Diffusion: Frequency and Attention Modulation for High-Resolution Image Generation with Stable Diffusion

Denoising Diffusion Bridge Models

Accelerated Image-Aware Generative Diffusion Modeling

Low-Bitwidth Floating Point Quantization for Efficient High-Quality Diffusion Models

Waving Goodbye to Low-Res: A Diffusion-Wavelet Approach for Image Super-Resolution

Diffusion Probabilistic Model Made Slim

Towards Extreme Image Compression with Latent Feature Guidance and Diffusion Prior

Binary Latent Diffusion