Abstract:Layer segmentation is important to quantitative analysis of retinal optical coherence tomography (OCT). Recently, deep learning based methods have been developed to automate this task and yield remarkable performance. However, due to the large spatial gap and potential mismatch between the B-scans of an OCT volume, all of them were based on 2D segmentation of individual B-scans, which may lose the continuity and diagnostic information of the retinal layers in 3D space. Besides, most of these methods required dense annotation of the OCT volumes, which is labor-intensive and expertise-demanding. This work presents a novel framework based on hybrid 2D-3D convolutional neural networks (CNNs) to obtain continuous 3D retinal layer surfaces from OCT volumes, which works well with both full and sparse annotations. The 2D features of individual B-scans are extracted by an encoder consisting of 2D convolutions. These 2D features are then used to produce the alignment displacement vectors and layer segmentation by two 3D decoders coupled via a spatial transformer module. Two losses are proposed to utilize the retinal layers' natural property of being smooth for B-scan alignment and layer segmentation, respectively, and are the key to the semi-supervised learning with sparse annotation. The entire framework is trained end-to-end. To the best of our knowledge, this is the first work that attempts 3D retinal layer segmentation in volumetric OCT images based on CNNs. Experiments on a synthetic dataset and three public clinical datasets show that our framework can effectively align the B-scans for potential motion correction, and achieves superior performance to state-of-the-art 2D deep learning methods in terms of both layer segmentation accuracy and cross-B-scan 3D continuity in both fully and semi-supervised settings, thus offering more clinical values than previous works.

Semantic Segmentation of Neuronal Bodies in Fluorescence Microscopy Using a 2D+3D CNN Training Strategy with Sparsely Annotated Data

Simultaneous Alignment and Surface Regression Using Hybrid 2D-3D Networks for 3D Coherent Layer Segmentation of Retinal OCT Images with Full and Annotations

3D Graph-S<SUP>2</SUP>Net: Shape-Aware Self-ensembling Network for Semi-supervised Segmentation with Bilateral Graph Convolution

A deep learning-based strategy for producing dense 3D segmentations from sparsely annotated 2D images

Neuron Segmentation Based on CNN with Semi-Supervised Regularization.

A 3D semantic segmentation network for accurate neuronal soma segmentation

3D Cnn-Based Soma Segmentation from Brain Images at Single-Neuron Resolution

A Neurocomputational Method for Fully Automated 3D Dendritic Spine Detection and Segmentation of Medium-Sized Spiny Neurons

Sparse Annotation is Sufficient for Bootstrapping Dense Segmentation

An Adaptive Sampling Scheme to Efficiently Train Fully Convolutional Networks for Semantic Segmentation

Cross-dimension Affinity Distillation for 3D EM Neuron Segmentation

Learning 3D Semantic Segmentation with only 2D Image Supervision

Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation

Self-supervised learning via inter-modal reconstruction and feature projection networks for label-efficient 3D-to-2D segmentation

Using Simulated Training Data of Voxel-Level Generative Models to Improve 3D Neuron Reconstruction

Neuron Image Segmentation via Learning Deep Features and Enhancing Weak Neuronal Structures

Self-Supervised Learning of Skeleton-Aware Morphological Representation for 3D Neuron Segments

Data Augmented 3D Semantic Scene Completion with 2D Segmentation Priors

3D Convolutional Neural Networks for Dendrite Segmentation Using Fine-Tuning and Hyperparameter Optimization

A 3D Convolutional Neural Network for Volumetric Image Semantic Segmentation

DDeep3M+: Adaptive Enhancement Powered Weakly Supervised Learning for Neuron Segmentation.