ExpRDiff: Short-exposure Guided Diffusion Model for Realistic Local Motion Deblurring

Zhongbao Yang,Jiangxin Dong,Jinhui Tang,Jinshan Pan
2024-12-12
Abstract:Removing blur caused by moving objects is challenging, as the moving objects are usually significantly blurry while the static background remains clear. Existing methods that rely on local blur detection often suffer from inaccuracies and cannot generate satisfactory results when focusing solely on blurred regions. To overcome these problems, we first design a context-based local blur detection module that incorporates additional contextual information to improve the identification of blurry regions. Considering that modern smartphones are equipped with cameras capable of providing short-exposure images, we develop a blur-aware guided image restoration method that utilizes sharp structural details from short-exposure images, facilitating accurate reconstruction of heavily blurred regions. Furthermore, to restore images realistically and visually-pleasant, we develop a short-exposure guided diffusion model that explores useful features from short-exposure images and blurred regions to better constrain the diffusion process. Finally, we formulate the above components into a simple yet effective network, named ExpRDiff. Experimental results show that ExpRDiff performs favorably against state-of-the-art methods.
Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: how to effectively remove local blurring caused by object motion, especially in photos taken in low - light environments, while maintaining the clarity of the static background. Existing methods have the following deficiencies when dealing with such problems: 1. **Traditional de - blurring methods**: They are mainly designed for global uniform or non - uniform blurring effects. When applied to locally blurred images, they will over - smooth the clear background, and the moving objects cannot be well restored. 2. **Methods based on object segmentation or blur detection**: In low - light conditions, these methods are difficult to accurately segment objects or detect blurred areas, thus affecting the de - blurring performance. 3. **Methods using short - exposure images as references**: Directly using short - exposure images as references cannot solve the problem well, because the quality of short - exposure images in some areas (such as clear areas in low - light environments) is poor. To solve the above problems, this paper proposes a new framework ExpRDiff, which aims to guide the diffusion model to perform more realistic image restoration by combining the information of short - exposure images and blurred areas. Specifically, this method contains the following modules: - **Context - based Local Blur Detection Module (CLBDM)**: By introducing additional context information to improve the identification accuracy of blurred areas. - **Blurry - aware Guided Image Restoration**: Using the information of short - exposure images and blurred areas, selectively process blurred areas and clear areas to achieve better image restoration. - **Short - exposure Guided Diffusion Model**: By fusing the features of short - exposure images and blurred areas, better constrain the diffusion process and generate more realistic restored images. Through the synergy of these modules, ExpRDiff can achieve better results when processing images with local blurring effects, especially in dynamic scenes photographed in low - light environments. ### Summary of Mathematical Formulas 1. **Calculation of confidence maps in the context - blur detection module**: \[ V_i = S\left(\frac{D(P_i)+G_i}{\tau}\right) \] where \( G_i = -\log(-\log(U_i)) \), \( U_i\sim\text{Uniform}(0, 1) \), \( S(\cdot) \) represents the Softmax operation, and \( \tau \) is a temperature parameter. 2. **Cross - entropy loss function**: \[ L = -\frac{1}{N}\sum_{i = 1}^N\left[V_i\log(M_{gt}^i)+(1 - V_i)\log(1 - M_{gt}^i)\right] \] 3. **Optimization of linear coefficients in blurry - aware guided image restoration**: \[ a_k=\frac{\sum_{i\in w_k}M(x_i)^2R(x_i)B(x_i)-\sum_{i\in w_k}M(x_i)^2R(x_i)b_k}{\sum_{i\in w_k}M(x_i)^2R(x_i)^2+\epsilon} \] \[ b_k=\frac{\sum_{i\in w_k}M(x_i)^2B(x_i)-\sum_{i\in w_k}M(x_i)^2a_kR(x_i)}{\sum_{i\in w_k}M(x_i)^2} \] 4. **Attention mechanism in the feature fusion module**: \[ D_i = S\left(\frac{Q(\tilde{D}_i)K(\tilde{D}_i)^\top}{\gamma}\right)V(\ha