Abstract:Recent advances in computer vision have led to significant progress in the generation of realistic image data, with denoising diffusion probabilistic models proving to be a particularly effective method. In this study, we demonstrate that diffusion models can effectively generate fully-annotated microscopy image data sets through an unsupervised and intuitive approach, using rough sketches of desired structures as the starting point. The proposed pipeline helps to reduce the reliance on manual annotations when training deep learning-based segmentation approaches and enables the segmentation of diverse datasets without the need for human annotations. We demonstrate that segmentation models trained with a small set of synthetic image data reach accuracy levels comparable to those of generalist models trained with a large and diverse collection of manually annotated image data, thereby offering a streamlined and specialized application of segmentation models. Modern generative techniques have unlocked the potential to create realistic image data of high quality, prompting the possibility of substituting real image data in segmentation training workflows. Our study highlights the capacity of denoising diffusion probabilistic models to generate high-quality microscopy image data. With adjustments to the generation process, these models can produce realistic fully-annotated image datasets through an intuitive and unsupervised approach. The parameters of the generative pipeline undergo optimization through various evaluations, resulting in synthetic image data that exhibits high PSNR scores. Our practical experiments encompass multiple scenarios, including manual annotations, initial segmentations, and simulations as starting points, demonstrating the versatility of our approach. Importantly, we compare the performance of segmentation models trained on a limited set of synthetic image data with those trained on a vast and diverse collection of manually annotated data, demonstrating the potential of our pipeline to alleviate the reliance on extensive manually annotated datasets. Our approach lays the groundwork for similar applications, thereby promoting the much-needed availability of publicly accessible fully-annotated image datasets and advancing the goal of annotation-free segmentation.

Seal2Real: Prompt Prior Learning on Diffusion Model for Unsupervised Document Seal Data Generation and Realisation

DatasetDM: Synthesizing Data with Perception Annotations Using Diffusion Models

Unleashing Text-to-Image Diffusion Models for Visual Perception

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models

DetDiffusion: Synergizing Generative and Perceptive Models for Enhanced Data Generation and Perception

DiffuMask: Synthesizing Images with Pixel-level Annotations for Semantic Segmentation Using Diffusion Models

Dataset Diffusion: Diffusion-based Synthetic Dataset Generation for Pixel-Level Semantic Segmentation

Seal: Advancing Speech Language Models to be Few-Shot Learners

Denoising diffusion probabilistic models for generation of realistic fully-annotated microscopy image datasets

Safe-SD: Safe and Traceable Stable Diffusion with Text Prompt Trigger for Invisible Generative Watermarking

Segment Any Point Cloud Sequences by Distilling Vision Foundation Models

Label-Efficient Semantic Segmentation with Diffusion Models

SEAL: A Framework for Systematic Evaluation of Real-World Super-Resolution

CAD: Photorealistic 3D Generation via Adversarial Distillation

Learning from Mistakes: Iterative Prompt Relabeling for Text-to-Image Diffusion Model Training

SEAL: Learning Heuristics for Community Detection with Generative Adversarial Networks

DB-EAC and LSTR: DBnet based seal text detection and Lightweight Seal Text Recognition

Denoising Diffusion Semantic Segmentation with Mask Prior Modeling

Prompting Diffusion Representations for Cross-Domain Semantic Segmentation

Diffusion Features to Bridge Domain Gap for Semantic Segmentation

A Novel Seal Imprint Verification Method Based On Analysis Of Difference Images And Symbolic Representation