Unsupervised Coordinate-Based Video Denoising

Mary Damilola Aiyetigbo,Dineshchandar Ravichandran,Reda Chalhoub,Peter Kalivas,Nianyi Li

2023-07-01

Abstract:In this paper, we introduce a novel unsupervised video denoising deep learning approach that can help to mitigate data scarcity issues and shows robustness against different noise patterns, enhancing its broad applicability. Our method comprises three modules: a Feature generator creating features maps, a Denoise-Net generating denoised but slightly blurry reference frames, and a Refine-Net re-introducing high-frequency details. By leveraging the coordinate-based network, we can greatly simplify the network structure while preserving high-frequency details in the denoised video frames. Extensive experiments on both simulated and real-captured demonstrate that our method can effectively denoise real-world calcium imaging video sequences without prior knowledge of noise models and data augmentation during training.

Computer Vision and Pattern Recognition,Image and Video Processing

What problem does this paper attempt to address?

This paper attempts to solve several key problems in video denoising:

1. **Data scarcity problem**: In video denoising tasks, it is very difficult to obtain a large number of labeled datasets (i.e., datasets containing both noisy frames and clean frames) especially in fields such as medical imaging. This limits the application of supervised learning methods.

2. **Robustness to different noise patterns**: Existing denoising methods perform differently when dealing with different types and complexities of noise and lack universal applicability. For example, some methods may work well when dealing with Gaussian noise but perform poorly when dealing with Poisson noise or impulse noise.

3. **Denoising of short - sequence videos**: In some application scenarios, such as microscope imaging, the video sequence may be very short, which makes self - supervised methods based on the "blind - spot" technique difficult to work effectively because these methods usually need a long video sequence to learn the noise model.

To address the above challenges, this paper proposes a new unsupervised video denoising method, which has the following features:

- **No need for labeled data**: Through unsupervised learning, this method can directly learn denoising from the original noisy video without prior knowledge of the noise model or data augmentation.

- **Simplified network structure**: By using a coordinate - based network, this method can simplify the network structure while retaining high - frequency details and improving the denoising effect.

- **Three - module architecture**:
- **Feature Generator**: Generate a feature map aligned with the input frame coordinates.
- **Denoise - Net**: Use the feature map to generate a denoised but slightly blurred reference frame.
- **Refine - Net**: Re - introduce high - frequency details to improve the clarity of the denoised frame.

- **Wide applicability**: This method performs well on multiple types of noise (such as Gaussian noise, Poisson noise, impulse noise) and video sequences of different lengths, and is especially suitable for denoising short - sequence videos.

Through these innovations, this paper aims to provide an efficient, robust and widely applicable video denoising solution, especially in cases of data scarcity and diverse noise types.