Abstract:Event cameras offer the advantages of low latency, high temporal resolution and HDR compared to conventional cameras. Due to the asynchronous and sparse nature of events, many existing algorithms cannot be directly applied, necessitating the reconstruction of intensity frames. However, existing reconstruction methods often result in artifacts and edge blurring due to noise and event accumulation. In this paper, we argue that the key to event-based image reconstruction is to enhance the edge information of objects and restore the artifacts in the reconstructed images. To explain, edge information is one of the most important features in the event stream, providing information on the shape and contour of objects. Considering the extraordinary capabilities of Denoising Diffusion Probabilistic Models (DDPMs) in image generation, reconstruction, and restoration, we propose a new framework which incorporate it into the reconstruction pipeline to obtain high-quality results which effectively remove artifacts and blur in reconstructed images. Specifically, we first extract edge information from the event stream using the proposed event-based denoising method. It employs the contrast maximization framework to remove noise from the event stream and extract clear object edge information. And then, the edge information is further adopted to our diffusion model, which is used to enhance the edges of objects in the reconstructed images, thus improving the restoration effect. Experimental results show that our method achieves significant improvements in the mean squared error (MSE), the structural similarity (SSIM), and the perceptual similarity (LPIPS) metrics, with average improvements of 40%, 15%, and 25%, respectively, compared to previous state-of-the-art models, and has good generalization performance.

DiracDiffusion: Denoising and Incremental Reconstruction with Assured Data-Consistency

Decoupled Data Consistency with Diffusion Purification for Image Restoration

Resfusion: Denoising Diffusion Probabilistic Models for Image Restoration Based on Prior Residual Noise

Event-Diffusion: Event-Based Image Reconstruction and Restoration with Diffusion Models

Denoising Monte Carlo Renders with Diffusion Models

High-Fidelity Diffusion-based Image Editing

Improving Diffusion-Based Image Restoration with Error Contraction and Error Correction

A Variational Perspective on Solving Inverse Problems with Diffusion Models

Efficient Diffusion Model for Image Restoration by Residual Shifting

Ambient Diffusion Posterior Sampling: Solving Inverse Problems with Diffusion Models trained on Corrupted Data

ADIR: Adaptive Diffusion for Image Reconstruction

Consistent Diffusion: Denoising Diffusion Model with Data-Consistent Training for Image Restoration

Denoising Diffusion Models for Plug-and-Play Image Restoration

Adapt and Diffuse: Sample-adaptive Reconstruction via Latent Diffusion Models

Enhancing Diffusion Posterior Sampling for Inverse Problems by Integrating Crafted Measurements

Diffusion Posterior Proximal Sampling for Image Restoration

Unleashing the Denoising Capability of Diffusion Prior for Solving Inverse Problems

Stimulating Diffusion Model for Image Denoising via Adaptive Embedding and Ensembling

Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervision