Abstract:Abstract The brain is a complex system that operates based on coordinated neuronal activities. Brain-wide cellular calcium imaging techniques have quickly advanced in recent years and become powerful tools for understanding the neuronal activities of small animal models. The whole brain imaging generally requires to extract the neuronal activities from three-dimensional (3D) image series. Unfortunately, the 3D image series are obtained under imaging conditions different among laboratories and extracting neuronal activities from the data requires multiple processes. Therefore researchers need to develop their own software, which has prevented the application of whole-brain imaging experiments in more laboratories. Here, we combined traditional image processing techniques with the powerful deep-learning method which can be flexibly modified to fit 3D image data in the nematode Caenorhabditis elegans obtained under different conditions. We first trained the 3D U-net deep network to classify each pixel into cell and non-cell categories. Cells merged as a whole region were further separated into individual cells by watershed segmentation. The cells were then tracked in 3D space over time with the combination of a feedforward network and a point set registration method to use local and global relative positions of the cells, respectively. Remarkably, one manually annotated 3D image combined with data augmentation was sufficient for training the deep networks to obtain satisfactory tracking results. Our method correctly tracked more than 98% of neurons in three different image datasets and successfully extracted brain-wide neuronal activities. Our method worked well even when the sampling rate was reduced: 86% correct in case 4/5 frames were removed, and when artificial noise was added into the raw images: 91% correct in case 35 times of background-level noise was added. Our results proved that deep learning is widely applicable to different datasets and can help us in establishing a flexible pipeline for extracting whole brain activities.

Practical method of cell segmentation in electron microscope image stack using deep convolutional neural network☆

Learn to segment single cells with deep distance estimator and deep cell detector

Deep residual contextual and subpixel convolution network for automated neuronal structure segmentation in micro-connectomics

Deep-learning-based flexible pipeline for segmenting and tracking cells in 3D image time series for whole brain imaging

Cross-dimension Affinity Distillation for 3D EM Neuron Segmentation

3D Cnn-Based Soma Segmentation from Brain Images at Single-Neuron Resolution

Automated Cell Structure Extraction for 3D Electron Microscopy by Deep Learning

Reducing manual operation time to obtain a segmentation learning model for volume electron microscopy using stepwise deep learning with manual correction

Automatic Mitochondria Segmentation for EM Data Using a 3D Supervised Convolutional Network

NCIS: Deep Color Gradient Maps Regression and Three-Class Pixel Classification for Enhanced Neuronal Cell Instance Segmentation in Nissl-Stained Histological Images

Joint Cell Nuclei Detection and Segmentation in Microscopy Images Using 3D Convolutional Networks

DeepSplit: Segmentation of Microscopy Images Using Multi-task Convolutional Networks

A deep learning-based strategy for producing dense 3D segmentations from sparsely annotated 2D images

A Reliable Distributed Convolutional Neural Network for Biology Image Segmentation.

Large Scale Image Segmentation with Structured Loss Based Deep Learning for Connectome Reconstruction

Real-time and accurate deep learning-based multi-organ nucleus segmentation in histology images

3D Consistent & Robust Segmentation of Cardiac Images by Deep Learning with Spatial Propagation

A novel deep learning-based 3D cell segmentation framework for future image-based disease detection

Deep learning based supervised semantic segmentation of Electron Cryo-Subtomograms

DDeep3M+: Adaptive Enhancement Powered Weakly Supervised Learning for Neuron Segmentation.

CNN-based Segmentation of Medical Imaging Data