Abstract:Abstract Background Convolutional neural network (CNN)–based methods have shown excellent performance in denoising and reconstruction of super-resolved structured illumination microscopy (SR-SIM) data. Therefore, CNN-based architectures have been the focus of existing studies. However, Swin Transformer, an alternative and recently proposed deep learning–based image restoration architecture, has not been fully investigated for denoising SR-SIM images. Furthermore, it has not been fully explored how well transfer learning strategies work for denoising SR-SIM images with different noise characteristics and recorded cell structures for these different types of deep learning–based methods. Currently, the scarcity of publicly available SR-SIM datasets limits the exploration of the performance and generalization capabilities of deep learning methods. Results In this work, we present SwinT-fairSIM, a novel method based on the Swin Transformer for restoring SR-SIM images with a low signal-to-noise ratio. The experimental results show that SwinT-fairSIM outperforms previous CNN-based denoising methods. Furthermore, as a second contribution, two types of transfer learning—namely, direct transfer and fine-tuning—were benchmarked in combination with SwinT-fairSIM and CNN-based methods for denoising SR-SIM data. Direct transfer did not prove to be a viable strategy, but fine-tuning produced results comparable to conventional training from scratch while saving computational time and potentially reducing the amount of training data required. As a third contribution, we publish four datasets of raw SIM images and already reconstructed SR-SIM images. These datasets cover two different types of cell structures, tubulin filaments and vesicle structures. Different noise levels are available for the tubulin filaments. Conclusion The SwinT-fairSIM method is well suited for denoising SR-SIM images. By fine-tuning, already trained models can be easily adapted to different noise characteristics and cell structures. Furthermore, the provided datasets are structured in a way that the research community can readily use them for research on denoising, super-resolution, and transfer learning strategies.

Memorizing Swin-Transformer Denoising Network for Diffusion Model

Vision Transformers for Single Image Dehazing

Image Denoising Via Multi-Scale Gated Fusion Network

Practical Blind Image Denoising via Swin-Conv-UNet and Data Synthesis

Efficient Image Deblurring Networks based on Diffusion Models

Swinv2-Imagen: Hierarchical Vision Transformer Diffusion Models for Text-to-Image Generation

An Efficient Dehazing Algorithm Based on the Fusion of Transformer and Convolutional Neural Network.

Practical Blind Denoising via Swin-Conv-UNet and Data Synthesis

Dilated Residual Encode-Decode Networks for Image Denoising

SUNet: Swin Transformer UNet for Image Denoising

StruNet: Perceptual and low-rank regularized transformer for medical image denoising.

SwinDenoising: A Local and Global Feature Fusion Algorithm for Infrared Image Denoising

Evaluation of Swin Transformer and knowledge transfer for denoising of super-resolution structured illumination microscopy data

NGST-Net: A N-Gram based Swin Transformer Network for improving multispectral and hyperspectral image fusion

DnSwin: Toward Real-World Denoising via Continuous Wavelet Sliding-Transformer

DnSwin: Toward real-world denoising via a continuous Wavelet Sliding Transformer

A Dynamic Network with Transformer for Image Denoising

Image Super-resolution Reconstruction Network based on Enhanced Swin Transformer via Alternating Aggregation of Local-Global Features

SwinVI:3D Swin Transformer Model with U-net for Video Inpainting.

Complex Image Generation SwinTransformer Network for Audio Denoising

Generative AI for Rapid Diffusion MRI with Improved Image Quality, Reliability and Generalizability