Abstract:The segmentation and tracking of living cells play a vital role within the biomedical domain, particularly in cancer research, drug development, and developmental biology. These are usually tedious and time-consuming tasks that are traditionally done by biomedical experts. Recently, to automatize these processes, deep learning based segmentation and tracking methods have been proposed. These methods require large-scale datasets and their full potential is constrained by the scarcity of annotated data in the biomedical imaging domain. To address this limitation, we propose Biomedical Video Diffusion Model (BVDM), capable of generating realistic-looking synthetic microscopy videos. Trained only on a single real video, BVDM can generate videos of arbitrary length with pixel-level annotations that can be used for training data-hungry models. It is composed of a denoising diffusion probabilistic model (DDPM) generating high-fidelity synthetic cell microscopy images and a flow prediction model (FPM) predicting the non-rigid transformation between consecutive video frames. During inference, initially, the DDPM imposes realistic cell textures on synthetic cell masks which are generated based on real data statistics. The flow prediction model predicts the flow field between consecutive masks and applies that to the DDPM output from the previous time frame to create the next one while keeping temporal consistency. BVDM outperforms state-of-the-art synthetic live cell microscopy video generation models. Furthermore, we demonstrate that a sufficiently large synthetic dataset enhances the performance of cell segmentation and tracking models compared to using a limited amount of available real data.

Neural Cell Video Synthesis via Optical-Flow Diffusion

Annotated Biomedical Video Generation using Denoising Diffusion Probabilistic Models and Flow Fields

SynCellFactory: Generative Data Augmentation for Cell Tracking

W-Cell-Net: Multi-frame Interpolation of Cellular Microscopy Videos

An Organism Starts with a Single Pix-Cell: A Neural Cellular Diffusion for High-Resolution Image Synthesis

Video Frame Synthesis using Deep Voxel Flow

Neural 3D Video Synthesis from Multi-view Video

Optical Flow Representation Alignment Mamba Diffusion Model for Medical Video Generation

Controllable Longer Image Animation with Diffusion Models

Cascaded Diffusion Models for 2D and 3D Microscopy Image Synthesis to Enhance Cell Segmentation

Neural Radiance Flow for 4D View Synthesis and Video Processing

SinFusion: Training Diffusion Models on a Single Image or Video

Learning Dynamic Textures for Neural Rendering of Human Actors

Neural Video Fields Editing

Lumiere: A Space-Time Diffusion Model for Video Generation

Video Diffusion Models

The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimation

Imagen Video: High Definition Video Generation with Diffusion Models

Neuromorphic Optical Flow and Real-time Implementation with Event Cameras

ViewFusion: Learning Composable Diffusion Models for Novel View Synthesis