Abstract:Multiplexed imaging technologies have made it possible to interrogate complex tumor microenvironments at sub-cellular resolution within their native spatial context. However, proper quantification of this complexity requires the ability to easily and accurately segment cells into their sub-cellular compartments. Within the supervised learning paradigm, deep learning based segmentation methods demonstrating human level performance have emerged. However, limited work has been done in developing such generalist methods within the label-free unsupervised context. Here we present an unsupervised segmentation (UNSEG) method that achieves deep learning level performance without requiring any training data. UNSEG leverages a Bayesian-like framework and the specificity of nucleus and cell membrane markers to construct an probability estimate of each pixel belonging to the nucleus, cell membrane, or background. It uses this estimate to segment each cell into its nuclear and cell-membrane compartments. We show that UNSEG is more internally consistent and better at generalizing to the complexity of tissue morphology than current deep learning methods. This allows UNSEG to unambiguously identify the cytoplasmic compartment of a cell, which we employ to demonstrate its use in an exemplar biological scenario. Within the UNSEG framework, we also introduce a new perturbed watershed algorithm capable of stably and automatically segmenting a cluster of cell nuclei into individual cell nuclei that increases the accuracy of classical watershed. Perturbed watershed can also be used as a standalone algorithm that researchers can incorporate within their supervised or unsupervised learning approaches to extend classical watershed, particularly in the multiplexed imaging context. Finally, as part of developing UNSEG, we have generated a high-quality annotated gastrointestinal tissue (GIT) dataset, which we anticipate will be useful for the broader research community. We demonstrate the efficacy of UNSEG on the GIT dataset, publicly available datasets, and on a range of practical scenarios. In these contexts, we also discuss the possibility of bias inherent in quantification of segmentation accuracy based on score. Segmentation, despite its long antecedents, remains a challenging problem, particularly in the context of tissue samples. UNSEG, an easy-to-use algorithm, provides an unsupervised approach to overcome this bottleneck, and as we discuss, can help improve deep learning based segmentation methods by providing a bridge between unsupervised and supervised learning paradigms.

Unsupervised segmentation of noisy electron microscopy images using salient watersheds and region merging

Hierarchical Level Features Based Trainable Segmentation for Electron Microscopy Images

SSHMT: Semi-supervised Hierarchical Merge Tree for Electron Microscopy Image Segmentation

Watershed Learning Merge Tree Used for Segmenting Neurons in SEM Images

Semi-Automatic Neuron Tracing in Electron Microscope Images: Preliminary Results

A modular hierarchical approach to 3D electron microscopy image segmentation.

Automated scalable segmentation of neurons from multispectral images

Automatically Segmenting and Reconstructing Neurons in SEM Images

Analyzing Image Segmentation for Connectomics

Addressing Distribution Mismatch for Effective Semi-Supervised Neuron Segmentation

UNSEG: unsupervised segmentation of cells and their nuclei in complex tissue samples

DEEP CONTEXTUAL RESIDUAL NETWORK FOR ELECTRON MICROSCOPY IMAGE SEGMENTATION IN CONNECTOMICS

A Hybrid Cnn-Rf Method for Electron Microscopy Images Segmentation

A Neurocomputational Method for Fully Automated 3D Dendritic Spine Detection and Segmentation of Medium-Sized Spiny Neurons

Semi-Supervised Segmentation of Mitochondria from Electron Microscopy Images Using Spatial Continuity

Large Scale Image Segmentation with Structured Loss Based Deep Learning for Connectome Reconstruction

Automatic Mitochondria Segmentation for EM Data Using a 3D Supervised Convolutional Network

Segmentation Of Mitochondria Based On Sem Images

Segmentation of neuronal structures using SARSA (λ)-based boundary amendment with reinforced gradient-descent curve shape fitting

An Iterative Convolutional Neural Network Algorithm Improves Electron Microscopy Image Segmentation