Star-convex Polyhedra for 3D Object Detection and Segmentation in Microscopy

Martin Weigert,Uwe Schmidt,Robert Haase,Ko Sugawara,Gene Myers

DOI: https://doi.org/10.1109/WACV45572.2020.9093435

2020-06-15

Abstract:Accurate detection and segmentation of cell nuclei in volumetric (3D) fluorescence microscopy datasets is an important step in many biomedical research projects. Although many automated methods for these tasks exist, they often struggle for images with low signal-to-noise ratios and/or dense packing of nuclei. It was recently shown for 2D microscopy images that these issues can be alleviated by training a neural network to directly predict a suitable shape representation (star-convex polygon) for cell nuclei. In this paper, we adopt and extend this approach to 3D volumes by using star-convex polyhedra to represent cell nuclei and similar shapes. To that end, we overcome the challenges of 1) finding parameter-efficient star-convex polyhedra representations that can faithfully describe cell nuclei shapes, 2) adapting to anisotropic voxel sizes often found in fluorescence microscopy datasets, and 3) efficiently computing intersections between pairs of star-convex polyhedra (required for non-maximum suppression). Although our approach is quite general, since star-convex polyhedra include common shapes like bounding boxes and spheres as special cases, our focus is on accurate detection and segmentation of cell nuclei. Finally, we demonstrate on two challenging datasets that our approach (StarDist-3D) leads to superior results when compared to classical and deep learning based methods.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

This paper attempts to solve the problem of accurately detecting and segmenting cell nuclei in three - dimensional fluorescence microscope images. Although there are already many automated detection and segmentation methods, these methods often perform poorly when dealing with images with low signal - to - noise ratios or densely arranged cell nuclei. This article proposes a new method based on star - convex polyhedra, called STARDIST - 3D, to improve these problems. Specifically, this method directly predicts the parameters of star - convex polyhedra suitable for describing the shape of cell nuclei by training a neural network, thereby achieving more accurate detection and segmentation. In addition, this method also addresses the following challenges:

1. **Finding a parameter - efficient star - convex polyhedral representation**: It can faithfully describe the shape of cell nuclei.
2. **Adapting to anisotropic voxel sizes**: This is a common problem in fluorescence microscope datasets.
3. **Efficiently calculating the intersections between star - convex polyhedra**: This is necessary for non - maximum suppression (NMS).

Through these improvements, STARDIST - 3D has shown results superior to classical methods and deep - learning - based methods on two challenging datasets.