EaDeblur-GS: Event assisted 3D Deblur Reconstruction with Gaussian Splatting

Yuchen Weng,Zhengwen Shen,Ruofan Chen,Qi Wang,Jun Wang

2024-09-05

Abstract:3D deblurring reconstruction techniques have recently seen significant advancements with the development of Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS). Although these techniques can recover relatively clear 3D reconstructions from blurry image inputs, they still face limitations in handling severe blurring and complex camera motion. To address these issues, we propose Event-assisted 3D Deblur Reconstruction with Gaussian Splatting (EaDeblur-GS), which integrates event camera data to enhance the robustness of 3DGS against motion blur. By employing an Adaptive Deviation Estimator (ADE) network to estimate Gaussian center deviations and using novel loss functions, EaDeblur-GS achieves sharp 3D reconstructions in real-time, demonstrating performance comparable to state-of-the-art methods.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: in the case of dealing with severe blurring and complex camera movements, the existing 3D deblurring and reconstruction techniques (such as NeRF and 3D Gaussian Splatting) still have limitations. Specifically:

1. **Limitations of existing methods**:
- When the input image is severely blurred, the existing 3D deblurring and reconstruction methods (such as RGB single - modal - based 3DGS and NeRF) can only produce relatively clear 2D rendering results, and are unable to reconstruct high - quality 3D objects.
- Existing methods perform poorly when dealing with extremely blurred or non - uniformly blurred images.
- Methods using only RGB images have limited effectiveness in complex camera movement scenarios.
- Although methods fusing event - camera data are helpful for improving the reconstruction quality, they still face challenges in estimating the camera movement trajectory.

2. **The proposed new method**:
- The paper proposes a new method named Event - assisted 3D Deblur Reconstruction with Gaussian Splatting (EaDeblur - GS), which enhances the robustness of 3D Gaussian Splatting to motion blur by combining event - camera data.
- EaDeblur - GS utilizes a novel Adaptive Deviation Estimator (ADE) network and two new loss functions (blur loss and event - integration loss) to achieve real - time, clear 3D reconstruction.

3. **Objectives**:
- Improve the quality of 3D reconstruction, especially when dealing with severe blurring and complex camera movements.
- Achieve real - time, high - quality 3D reconstruction to ensure efficiency and accuracy in practical applications.

In summary, this paper aims to solve the deficiencies of existing 3D deblurring and reconstruction techniques in dealing with severely blurred and complex motion scenarios by introducing event - camera data and improved algorithms, thereby achieving higher - quality and more efficient 3D reconstruction.