Abstract:We present an approach to jointly detect mitotic events spatially and temporally in time-lapse phase contrast microscopy images. In particular, we combine a convolutional neural network (CNN) and a long short-term memory (LSTM) network to detect mitotic events in patch sequences. The CNN-LSTM network can be trained end-to-end to simultaneously learn convolutional features within each frame and temporal dynamics between frames, without hand-crafted visual or temporal feature design. Owing to the LSTM layer, this approach is able to detect mitotic events in patch sequences of variable length, as well as making use of longer context information among frames in the sequences. To the best of our knowledge, this is the first work to detect mitosis using deep learning in both spatial and temporal domains. Experiments have shown that the CNN-LSTM network can be trained efficiently, and we evaluate this design by applying the network to original raw microscopy image sequences to locate mitotic events both spatially and temporally. The data with which we validate the proposed method include C3H10 mesenchymal and C2C12 myoblastic stem cell populations. Our approach achieved the F score of 98.72% on the C2C12 data set, and the F score of 96.5% on the C3H10 data set. The results on both data sets outperform the traditional graph modelbased approaches by a large margin, both in terms of detection accuracy and frame localization accuracy. Furthermore, we have developed a framework to aid humans in annotating mitosis with high efficiency and accuracy in raw phase contrast microscopy images based on the joint detection results using the proposed method. Under this framework, expert level annotations can be obtained in raw phase contrast microscopy image sequences, and the annotations have shown to further improve the training performance of the CNN-LSTM network.

Automatic Cell Cycle Localization Using Latent-Dynamic Conditional Random Fields

Multi-Grained Random Fields for Mitosis Identification in Time-Lapse Phase Contrast Microscopy Image Sequences.

Accurate and Fast Mitotic Detection Using an Anchor-Free Method Based on Full-Scale Connection with Recurrent Deep Layer Aggregation in 4D Microscopy Images

Cell mitosis event analysis in phase contrast microscopy images using deep learning.

Label-free cell cycle analysis for high-throughput imaging flow cytometry

Cell Type-Independent Mitosis Event Detection Via Hidden-State Conditional Neural Fields

Deep Reinforcement Learning-Based Progressive Sequence Saliency Discovery Network for Mitosis Detection in Time-Lapse Phase-Contrast Microscopy Images

Robust Plant Cell Tracking in Noisy Image Sequences Using Optimal CRF Graph Matching

A Fuzzy Segmentation Method to Learn Classification of Mitosis

Spatiotemporal Joint Mitosis Detection Using CNN-LSTM Network in Time-Lapse Phase Contrast Microscopy Images.

A Deep Local Patch Matching Network for Cell Tracking in Microscopy Image Sequences Without Registration

3D Convolutional Networks-Based Mitotic Event Detection in Time-Lapse Phase Contrast Microscopy Image Sequences of Stem Cell Populations

Predicting cell cycle stage from 3D single-cell nuclear-stained images

Application of Local Activity Theory of CNN to the Coupled Cell Cycle Clock Model

CHARACTERIZATION OF SINGLE CELL DYNAMIC MORPHOLOGY BY LOCAL DEFORMATION PATTERN MODELING

Cell Mitosis Detection Using Deep Neural Networks

Cell Tracking Across Noisy Image Sequences Via Faster R-CNN and Dynamic Local Graph Matching

Exploring Contextual Relationships for Cervical Abnormal Cell Detection

Analyzing Dynamic Cellular Morphology in Time-Lapsed Images Enabled by Cellular Deformation Pattern Recognition.

A Multi-Seed Dynamic Local Graph Matching Model for Tracking of Densely Packed Cells Across Unregistered Microscopy Image Sequences

Sequential Saliency Guided Deep Neural Network for Joint Mitosis Identification and Localization in Time-Lapse Phase Contrast Microscopy Images