Abstract:Abstract The brain is a complex system that operates based on coordinated neuronal activities. Brain-wide cellular calcium imaging techniques have quickly advanced in recent years and become powerful tools for understanding the neuronal activities of small animal models. The whole brain imaging generally requires to extract the neuronal activities from three-dimensional (3D) image series. Unfortunately, the 3D image series are obtained under imaging conditions different among laboratories and extracting neuronal activities from the data requires multiple processes. Therefore researchers need to develop their own software, which has prevented the application of whole-brain imaging experiments in more laboratories. Here, we combined traditional image processing techniques with the powerful deep-learning method which can be flexibly modified to fit 3D image data in the nematode Caenorhabditis elegans obtained under different conditions. We first trained the 3D U-net deep network to classify each pixel into cell and non-cell categories. Cells merged as a whole region were further separated into individual cells by watershed segmentation. The cells were then tracked in 3D space over time with the combination of a feedforward network and a point set registration method to use local and global relative positions of the cells, respectively. Remarkably, one manually annotated 3D image combined with data augmentation was sufficient for training the deep networks to obtain satisfactory tracking results. Our method correctly tracked more than 98% of neurons in three different image datasets and successfully extracted brain-wide neuronal activities. Our method worked well even when the sampling rate was reduced: 86% correct in case 4/5 frames were removed, and when artificial noise was added into the raw images: 91% correct in case 35 times of background-level noise was added. Our results proved that deep learning is widely applicable to different datasets and can help us in establishing a flexible pipeline for extracting whole brain activities.

Intra- and Inter-Cellular Awareness for 3D Neuron Tracking and Segmentation in Large-Scale Connectomics.

An Efficient Method for Neuronal Tracking in Electron Microscopy Images

Deep-learning-based flexible pipeline for segmenting and tracking cells in 3D image time series for whole brain imaging

Semi-Automatic Neuron Tracing in Electron Microscope Images: Preliminary Results

NeuroSeg-II: A Deep Learning Approach for Generalized Neuron Segmentation in Two-Photon Ca2+ Imaging

Cross-dimension Affinity Distillation for 3D EM Neuron Segmentation

NeuroSeg: Automated Cell Detection and Segmentation for in Vivo Two-Photon Ca 2+ Imaging Data

DEEP CONTEXTUAL RESIDUAL NETWORK FOR ELECTRON MICROSCOPY IMAGE SEGMENTATION IN CONNECTOMICS

Deep residual contextual and subpixel convolution network for automated neuronal structure segmentation in micro-connectomics

Learning Multimodal Volumetric Features for Large-Scale Neuron Tracing

NeuroSeg-III: Efficient Neuron Segmentation in Two-Photon Ca2+ Imaging Data Using Self-Supervised Learning

Large Scale Image Segmentation with Structured Loss Based Deep Learning for Connectome Reconstruction

Brain Image Segmentation for Ultrascale Neuron Reconstruction via an Adaptive Dual-Task Learning Network

A modular hierarchical approach to 3D electron microscopy image segmentation.

A modular framework for multi-scale tissue imaging and neuronal segmentation

Deep learning and shapes similarity for joint segmentation and tracing single neurons in SEM images.

3D Neuron Microscopy Image Segmentation Via the Ray-Shooting Model and a DC-BLSTM Network.

An automated three-dimensional detection and segmentation method for touching cells by integrating concave points clustering and random walker algorithm.

UniSPAC: A Unified Segmentation Framework for Proofreading and Annotation in Connectomics

An effective AI integrated system for neuron tracing on anisotropic electron microscopy volume

Effective automated pipeline for 3D reconstruction of synapses based on deep learning