Abstract:Recent advances in computer vision have led to significant progress in the generation of realistic image data, with denoising diffusion probabilistic models proving to be a particularly effective method. In this study, we demonstrate that diffusion models can effectively generate fully-annotated microscopy image data sets through an unsupervised and intuitive approach, using rough sketches of desired structures as the starting point. The proposed pipeline helps to reduce the reliance on manual annotations when training deep learning-based segmentation approaches and enables the segmentation of diverse datasets without the need for human annotations. We demonstrate that segmentation models trained with a small set of synthetic image data reach accuracy levels comparable to those of generalist models trained with a large and diverse collection of manually annotated image data, thereby offering a streamlined and specialized application of segmentation models. Modern generative techniques have unlocked the potential to create realistic image data of high quality, prompting the possibility of substituting real image data in segmentation training workflows. Our study highlights the capacity of denoising diffusion probabilistic models to generate high-quality microscopy image data. With adjustments to the generation process, these models can produce realistic fully-annotated image datasets through an intuitive and unsupervised approach. The parameters of the generative pipeline undergo optimization through various evaluations, resulting in synthetic image data that exhibits high PSNR scores. Our practical experiments encompass multiple scenarios, including manual annotations, initial segmentations, and simulations as starting points, demonstrating the versatility of our approach. Importantly, we compare the performance of segmentation models trained on a limited set of synthetic image data with those trained on a vast and diverse collection of manually annotated data, demonstrating the potential of our pipeline to alleviate the reliance on extensive manually annotated datasets. Our approach lays the groundwork for similar applications, thereby promoting the much-needed availability of publicly accessible fully-annotated image datasets and advancing the goal of annotation-free segmentation.

Self-Supervised Learning with Generative Adversarial Networks for Electron Microscopy

High-throughput, high-resolution registration-free generated adversarial network microscopy

Two-stage Generative Models of Simulating Training Data at the Voxel Level for Large-Scale Microscopy Bioimage Segmentation

R2D2-GAN: Robust Dual Discriminator Generative Adversarial Network for Microscopy Hyperspectral Image Super-Resolution

Graph Self-Supervised Learning for Optoelectronic Properties of Organic Semiconductors

Exploiting Generative Self-Supervised Learning For The Assessment of Biological Images With Lack of Annotations: A COVID-19 Case-Study

Deep Generative Models-Assisted Automated Labeling for Electron Microscopy Images Segmentation

Super resolution-based methodology for self-supervised segmentation of microscopy images

Leveraging generative adversarial networks to create realistic scanning transmission electron microscopy images

Deep Learning Supersampled Scanning Transmission Electron Microscopy

Partial Scanning Transmission Electron Microscopy with Deep Learning

Unsupervised Domain Adaptation for the Histopathological Cell Segmentation Through Self-Ensembling

Efficient end-to-end learning for cell segmentation with machine generated weak annotations

Super-resolving microscopy images of Li-ion electrodes for fine-feature quantification using generative adversarial networks

Learning with limited target data to detect cells in cross-modality images

Denoising diffusion probabilistic models for generation of realistic fully-annotated microscopy image datasets

Improving Unsupervised Stain-To-Stain Translation using Self-Supervision and Meta-Learning

Artificial Scanning Electron Microscopy Images Created by Generative Adversarial Networks from Simulated Particle Assemblies

μPIX: Leveraging Generative AI for Enhanced, Personalized and Sustainable Microscopy

Using CycleGANs to Generate Realistic STEM Images for Machine Learning