Abstract:Background: Spatial mapping of transcriptional states provides valuable biological insights into cellular functions and interactions in the context of the tissue. Accurate 3D cell segmentation is a critical step in the analysis of this data towards understanding diseases and normal development in situ. Current approaches designed to automate 3D segmentation include stitching masks along one dimension, training a 3D neural network architecture from scratch, and reconstructing a 3D volume from 2D segmentations on all dimensions. However, the applicability of existing methods is hampered by inaccurate segmentations along the non-stitching dimensions, the lack of high-quality diverse 3D training data, and inhomogeneity of image resolution along orthogonal directions due to acquisition constraints; as a result, they have not been widely used in practice. Methods: To address these challenges, we formulate the problem of finding cell correspondence across layers with a novel optimal transport (OT) approach. We propose CellStitch, a flexible pipeline that segments cells from 3D images without requiring large amounts of 3D training data. We further extend our method to interpolate internal slices from highly anisotropic cell images to recover isotropic cell morphology. Results: We evaluated the performance of CellStitch through eight 3D plant microscopic datasets with diverse anisotropic levels and cell shapes. CellStitch substantially outperforms the state-of-the art methods on anisotropic images, and achieves comparable segmentation quality against competing methods in isotropic setting. We benchmarked and reported 3D segmentation results of all the methods with instance-level precision, recall and average precision (AP) metrics. Conclusions: The proposed OT-based 3D segmentation pipeline outperformed the existing state-of-the-art methods on different datasets with nonzero anisotropy, providing high fidelity recovery of 3D cell morphology from microscopic images.

Learning Neuron Stitching for Connectomics

Fast and Lightweight Network Improves Serial Brain Section Stitching

RecStitchNet: Learning to Stitch Images with Rectangular Boundaries

Deep residual contextual and subpixel convolution network for automated neuronal structure segmentation in micro-connectomics

Large Scale Image Segmentation with Structured Loss Based Deep Learning for Connectome Reconstruction

Learning Multimodal Volumetric Features for Large-Scale Neuron Tracing

Neuron Image Segmentation via Learning Deep Features and Enhancing Weak Neuronal Structures

MosaicNet: A deep-learning-based multi-tile biomedical image stitching method

FusionNet: A Deep Fully Residual Convolutional Neural Network for Image Segmentation in Connectomics

Towards Computational Analytics of 3D Neuron Images Using Deep Adversarial Learning

Cross-dimension Affinity Distillation for 3D EM Neuron Segmentation

A hierarchically annotated dataset drives tangled filament recognition in digital neuron reconstruction

Graph Convolution Based Cross-Network Multiscale Feature Fusion for Deep Vessel Segmentation

Neuron segmentation using 3D wavelet integrated encoder-decoder network

DDeep3M+: Adaptive Enhancement Powered Weakly Supervised Learning for Neuron Segmentation.

Cellstitch: 3D cellular anisotropic image segmentation via optimal transport

DINeuro: Distilling Knowledge from 2D Natural Images via Deformable Tubular Transferring Strategy for 3D Neuron Reconstruction

Graph-based cell pattern recognition for merging the multi-modal optical microscopic image of neurons

NerveStitcher2.0: Evolution of Stitching Algorithm for Corneal Confocal Microscope Images with Optical Flow.

NerveStitcher: Corneal Confocal Microscope Images Stitching with Neural Networks

3D Cnn-Based Soma Segmentation from Brain Images at Single-Neuron Resolution