CRiM-GS: Continuous Rigid Motion-Aware Gaussian Splatting from Motion-Blurred Images

Jungho Lee,Donghyeong Kim,Dogyoon Lee,Suhwan Cho,Minhyeok Lee,Sangyoun Lee

2024-12-08

Abstract:3D Gaussian Splatting (3DGS) has gained significant attention for their high-quality novel view rendering, motivating research to address real-world challenges. A critical issue is the camera motion blur caused by movement during exposure, which hinders accurate 3D scene reconstruction. In this study, we propose CRiM-GS, a \textbf{C}ontinuous \textbf{Ri}gid \textbf{M}otion-aware \textbf{G}aussian \textbf{S}platting that reconstructs precise 3D scenes from motion-blurred images while maintaining real-time rendering speed. Considering the complex motion patterns inherent in real-world camera movements, we predict continuous camera trajectories using neural ordinary differential equations (ODE). To ensure accurate modeling, we employ rigid body transformations with proper regularization, preserving object shape and size. Additionally, we introduce an adaptive distortion-aware transformation to compensate for potential nonlinear distortions, such as rolling shutter effects, and unpredictable camera movements. By revisiting fundamental camera theory and leveraging advanced neural training techniques, we achieve precise modeling of continuous camera trajectories. Extensive experiments demonstrate state-of-the-art performance both quantitatively and qualitatively on benchmark datasets.

Computer Vision and Pattern Recognition,Artificial Intelligence

What problem does this paper attempt to address?

### What problem does this paper attempt to solve?

This paper aims to solve the **3D scene reconstruction problem in camera - motion - blurred images**. Specifically, the authors propose a new framework named CRiM - GS (Continuous Rigid Motion - Aware Gaussian Splatting) for reconstructing accurate 3D scenes from motion - blurred images while maintaining real - time rendering speed.

#### Problem background

In the real world, due to the movement of the camera during exposure, images may be motion - blurred, which poses a challenge to accurate 3D scene reconstruction. Existing methods such as NeRF and 3D Gaussian Splatting (3DGS) rely on clear input images, which usually require very small aperture settings to obtain a large depth of field, but this will limit the light intake and lead to a longer exposure time, thus introducing complex motion blur.

#### Solution

To meet this challenge, CRiM - GS improves existing methods in the following ways:

1. **Continuous rigid - body motion modeling**: Use neural ordinary differential equations (Neural ODE) to model the continuous motion trajectory of the camera during the exposure time. This method can capture complex motion patterns and ensure that the model can handle nonlinear distortions in practical applications.

2. **Rigid - body transformation and regularization**: Combine rigid - body transformation and add appropriate regularization to keep the shape and size of objects unchanged. This helps to accurately capture the geometric consistency of static objects during camera movement.

3. **Adaptive distortion - aware transformation**: Introduce an adaptive distortion - aware transformation to compensate for nonlinear distortions (such as rolling shutter effects) that may occur during fast camera movement. This transformation increases the flexibility of the model, enabling it to better handle complex real - scene situations.

4. **Pixel - level weighted summation**: Generate the final motion - blurred image by performing pixel - level weighted summation on multiple rendered images. This method allows the model to learn the weight distribution between different images, thereby improving the reconstruction quality.

Through the above methods, CRiM - GS can achieve high - precision 3D scene reconstruction when dealing with motion - blurred images and has achieved state - of - the - art performance on benchmark datasets.

#### Experimental results

The paper conducted extensive experiments on synthetic and real - world scene datasets. The results show that CRiM - GS outperforms other methods in terms of metrics such as peak signal - to - noise ratio (PSNR), structural similarity index (SSIM), and learned perceptual image patch similarity (LPIPS). In particular, for the LPIPS metric, CRiM - GS has improved by approximately 52% and 33% on synthetic and real - scene datasets respectively.

In summary, by proposing the CRiM - GS framework, this paper solves the 3D scene reconstruction problem in motion - blurred images and significantly improves the reconstruction quality and application range.