Domain-adaptive Video Deblurring via Test-time Blurring

Jin-Ting He,Fu-Jen Tsai,Jia-Hao Wu,Yan-Tsung Peng,Chung-Chi Tsai,Chia-Wen Lin,Yen-Yu Lin

2024-07-12

Abstract:Dynamic scene video deblurring aims to remove undesirable blurry artifacts captured during the exposure process. Although previous video deblurring methods have achieved impressive results, they suffer from significant performance drops due to the domain gap between training and testing videos, especially for those captured in real-world scenarios. To address this issue, we propose a domain adaptation scheme based on a blurring model to achieve test-time fine-tuning for deblurring models in unseen domains. Since blurred and sharp pairs are unavailable for fine-tuning during inference, our scheme can generate domain-adaptive training pairs to calibrate a deblurring model for the target domain. First, a Relative Sharpness Detection Module is proposed to identify relatively sharp regions from the blurry input images and regard them as pseudo-sharp images. Next, we utilize a blurring model to produce blurred images based on the pseudo-sharp images extracted during testing. To synthesize blurred images in compliance with the target data distribution, we propose a Domain-adaptive Blur Condition Generation Module to create domain-specific blur conditions for the blurring model. Finally, the generated pseudo-sharp and blurred pairs are used to fine-tune a deblurring model for better performance. Extensive experimental results demonstrate that our approach can significantly improve state-of-the-art video deblurring methods, providing performance gains of up to 7.54dB on various real-world video deblurring datasets. The source code is available at <a class="link-external link-https" href="https://github.com/Jin-Ting-He/DADeblur" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper attempts to address the issue of domain adaptation in video deblurring, particularly the blurring caused by camera shake or moving objects in videos captured in real-world scenarios. Although previous video deblurring methods have achieved significant results, their performance drops significantly when there is a domain gap between the training data and the test data. To solve this problem, the authors propose a domain adaptation scheme based on a blur model, aiming to improve the performance of the deblurring model in unseen domains through test-time fine-tuning.

Specifically, the main contributions of the paper include:
1. Proposing a test-time domain adaptation method that uses ID-Blau to generate domain-specific blur conditions, enabling test-time fine-tuning of the deblurring model.
2. Designing a Relative Sharpness Detection Module (RSDM) to extract relatively sharp regions from blurry videos as pseudo-sharp images.
3. Introducing a Domain Adaptation Blur Condition Generation Module (DBCGM) to generate domain-specific blur conditions to blur those pseudo-sharp images.
4. Experimental results show that this method can significantly improve the performance of existing state-of-the-art video deblurring models on multiple real-world video deblurring datasets, with a maximum improvement of up to 7.54dB.

Through these techniques, the paper successfully addresses the issue of domain adaptation in video deblurring, enhancing the robustness and generalization ability of the model in practical applications.