Abstract:In this paper, we introduce a novel unsupervised network to denoise microscopy videos featured by image sequences captured by a fixed location microscopy camera. Specifically, we propose a DeepTemporal Interpolation method, leveraging a temporal signal filter integrated into the bottom CNN layers, to restore microscopy videos corrupted by unknown noise types. Our unsupervised denoising architecture is distinguished by its ability to adapt to multiple noise conditions without the need for pre-existing noise distribution knowledge, addressing a significant challenge in real-world medical applications. Furthermore, we evaluate our denoising framework using both real microscopy recordings and simulated data, validating our outperforming video denoising performance across a broad spectrum of noise scenarios. Extensive experiments demonstrate that our unsupervised model consistently outperforms state-of-the-art supervised and unsupervised video denoising techniques, proving especially effective for microscopy videos.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is unsupervised denoising in fixed - position microscope videos. Specifically, the authors propose a method named **DeepTemporal Interpolation**, aiming to restore microscope videos contaminated by unknown noise types. This method is achieved by integrating a time - signal filter in the underlying CNN layers and can adapt to multiple noise conditions without prior knowledge of the noise distribution, thus solving a significant challenge in practical medical applications. ### Main Problems and Challenges 1. **Diverse Noise Sources**: The noise in microscope videos comes from multiple sources, such as photon scattering noise, background fluorescence, and detector electronic noise. These noises are not only of different types (Poisson noise and Gaussian noise) but also their intensities change with frames. 2. **Fixed - position Shooting**: Since microscope videos are usually shot at fixed positions, there is a lack of motion information in the videos, which makes it difficult for traditional motion - based denoising methods to be effectively applied. 3. **Variable Frame Rates**: The frame rates of microscope videos vary greatly, especially in calcium imaging recordings. This requires the denoising method to be able to handle these temporal changes without compromising the integrity of the observed neural signals. 4. **Unsupervised Learning**: Many of the existing state - of - the - art denoising methods rely on supervised learning and require a large number of noisy/clean image pairs for training. However, it is unrealistic to obtain clean reference images in medical imaging. ### Solutions The authors propose an unsupervised denoising method, which mainly consists of two main components: - **Feature Generator (Feature Generator \( G_\phi \))**: Use three deep convolutional layers to extract feature maps. - **Denoiser (Denoiser \( D_\theta \))**: Use the U - Net architecture to generate the denoised central frame. The key to the **DeepTemporal Interpolation** method is the introduction of a time filter, which assigns weights to the feature maps generated by the feature generator. By adjusting the weights of the feature maps, the denoising effect is improved. Specifically, the time filter will assign higher weights to feature maps far from the central frame and lower weights to those close to the central frame, thus avoiding the problem of rapid convergence to noisy content due to inter - frame redundancy in high - frame - rate videos. ### Experimental Verification The authors verified the effectiveness of this method through multiple datasets, including real microscope videos and simulated data. The experimental results show that this method performs well under various noise conditions. In particular, when dealing with videos of high frame rates and slow - moving objects, it outperforms existing supervised and unsupervised denoising methods. ### Main Contributions 1. **Proposed a New Unsupervised Video Denoising Method**: By applying a time filter on the feature maps, the over - fitting problem common in unsupervised models is effectively alleviated. 2. **Conducted a Comprehensive Evaluation on Real and Simulated Medical Imaging Datasets**: The results show that this model outperforms supervised and unsupervised methods in denoising performance. 3. **Made Significant Progress in Video Denoising for Non - Gaussian Noise Types**: This is a problem that existing CNN - based models have difficulty dealing with. In conclusion, this paper proposes an innovative unsupervised microscope video denoising method, which can effectively denoise under multiple noise conditions and has important practical application value.

Unsupervised Microscopy Video Denoising

Unsupervised Coordinate-Based Video Denoising

Unsupervised Deep Video Denoising with Untrained Network

Unsupervised Training of a Dynamic Context-Aware Deep Denoising Framework for Low-Dose Fluoroscopic Imaging

Deep Convolutional Architecture for Natural Image Denoising

Unsupervised Denoising for Signal-Dependent and Row-Correlated Imaging Noise

Temporal As a Plugin: Unsupervised Video Denoising with Pre-Trained Image Denoisers

Deep Unfolding Network for Efficient Mixed Video Noise Removal.

An unsupervised deep learning framework for medical image denoising

First image then video: A two-stage network for spatiotemporal video denoising

UNet-Att: a Self-Supervised Denoising and Recovery Model for Two-Photon Microscopic Image

Deep Joint Denoising and Detection for Enhanced Intracellular Particle Analysis

An Unsupervised Deep Learning Approach for Real-World Image Denoising.

Fluorescence Microscopy Images Denoising Via Deep Convolutional Sparse Coding

Live 4D-OCT denoising with self-supervised deep learning

Deformed2Self: Self-Supervised Denoising for Dynamic Medical Imaging

Stabilize, Decompose, and Denoise: Self-Supervised Fluoroscopy Denoising

FM2S: Self-Supervised Fluorescence Microscopy Denoising With Single Noisy Image

Blind microscopy image denoising with a deep residual and multiscale encoder/decoder network

Deep learning for fast denoising filtering in ultrasound localization microscopy

3D-B2U: Self-supervised Fluorescent Image Sequences Denoising.