Abstract:Automatic image segmentation is critical to scale up electron microscope (EM) connectome reconstruction. To this end, segmentation competitions, such as CREMI and SNEMI, exist to help researchers evaluate segmentation algorithms with the goal of improving them. Because generating ground truth is time-consuming, these competitions often fail to capture the challenges in segmenting larger datasets required in connectomics. More generally, the common metrics for EM image segmentation do not emphasize impact on downstream analysis and are often not very useful for isolating problem areas in the segmentation. For example, they do not capture connectivity information and often over-rate the quality of a segmentation as we demonstrate later. To address these issues, we introduce a novel strategy to enable evaluation of segmentation at large scales both in a supervised setting, where ground truth is available, or an unsupervised setting. To achieve this, we first introduce new metrics more closely aligned with the use of segmentation in downstream analysis and reconstruction. In particular, these include synapse connectivity and completeness metrics that provide both meaningful and intuitive interpretations of segmentation quality as it relates to the preservation of neuron connectivity. Also, we propose measures of segmentation correctness and completeness with respect to the percentage of "orphan" fragments and the concentrations of self-loops formed by segmentation failures, which are helpful in analysis and can be computed without ground truth. The introduction of new metrics intended to be used for practical applications involving large datasets necessitates a scalable software ecosystem, which is a critical contribution of this paper. To this end, we introduce a scalable, flexible software framework that enables integration of several different metrics and provides mechanisms to evaluate and debug differences between segmentations. We also introduce visualization software to help users to consume the various metrics collected. We evaluate our framework on two relatively large public groundtruth datasets providing novel insights on example segmentations.

UniSPAC: A Unified Segmentation Framework for Proofreading and Annotation in Connectomics

Semi-Automatic Neuron Tracing in Electron Microscope Images: Preliminary Results

Neuproofreader: an Interactive Proofreading System with Suggestive Prompts for Connectomics

NEURD offers automated proofreading and feature extraction for connectomics

Analyzing Image Segmentation for Connectomics

A Neurocomputational Method for Fully Automated 3D Dendritic Spine Detection and Segmentation of Medium-Sized Spiny Neurons

mEMbrain: an interactive deep learning MATLAB tool for connectomic segmentation on commodity desktops

Segmentor: a tool for manual refinement of 3D microscopy annotations

A Unified Framework for 3D Scene Understanding

PSSD-Transformer: Powerful Sparse Spike-Driven Transformer for Image Semantic Segmentation

PyTorch Connectomics: A Scalable and Flexible Segmentation Framework for EM Connectomics

A modular framework for multi-scale tissue imaging and neuronal segmentation

Deep residual contextual and subpixel convolution network for automated neuronal structure segmentation in micro-connectomics

Spider: A Unified Framework for Context-dependent Concept Segmentation

Automated neuropil segmentation of fluorescent images for Drosophila brains

Large Scale Image Segmentation with Structured Loss Based Deep Learning for Connectome Reconstruction

A modular hierarchical approach to 3D electron microscopy image segmentation.

Semi-Supervised Neuron Segmentation via Reinforced Consistency Learning

Learning Multimodal Volumetric Features for Large-Scale Neuron Tracing

UN-SAM: Universal Prompt-Free Segmentation for Generalized Nuclei Images

Large-scale EM Analysis of the Drosophila Antennal Lobe with Automatically Computed Synapse Point Clouds