Deep Neural Network for 3D Surface Segmentation based on Contour Tree Hierarchy

Wenchong He,Arpan Man Sainju,Zhe Jiang,Da Yan

DOI: https://doi.org/10.48550/arXiv.2008.11269

2020-08-26

Abstract:Given a 3D surface defined by an elevation function on a 2D grid as well as non-spatial features observed at each pixel, the problem of surface segmentation aims to classify pixels into contiguous classes based on both non-spatial features and surface topology. The problem has important applications in hydrology, planetary science, and biochemistry but is uniquely challenging for several reasons. First, the spatial extent of class segments follows surface contours in the topological space, regardless of their spatial shapes and directions. Second, the topological structure exists in multiple spatial scales based on different surface resolutions. Existing widely successful deep learning models for image segmentation are often not applicable due to their reliance on convolution and pooling operations to learn regular structural patterns on a grid. In contrast, we propose to represent surface topological structure by a contour tree skeleton, which is a polytree capturing the evolution of surface contours at different elevation levels. We further design a graph neural network based on the contour tree hierarchy to model surface topological structure at different spatial scales. Experimental evaluations based on real-world hydrological datasets show that our model outperforms several baseline methods in classification accuracy.

Computer Vision and Pattern Recognition,Machine Learning

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to classify pixels according to non - spatial features and surface topology given a 3D surface defined by an elevation function on a 2D grid and non - spatial features observed for each pixel. Specifically, the goal is to classify pixels into contiguous classes whose spatial extents follow the changes of the surface contour in the topological space, regardless of their spatial shapes and orientations. This problem has important applications in fields such as hydrology, planetary science, and biochemistry, but at the same time faces several unique challenges: 1. **The spatial extents of class segments follow the topological structure of the surface contour**: This violates the assumption of traditional convolutional neural networks (CNNs) that the spatial structure of class segments is regular on the Euclidean plane. 2. **The topological structure exists at multiple spatial scales**: Surfaces at different resolutions will show different local fluctuations and global trends. Existing widely successful deep - learning models usually rely on convolution and pooling operations to learn regular structural patterns on the grid and are therefore not suitable for this problem. To solve these problems, the authors propose a graph neural network model based on the contour - tree hierarchy. By representing the surface topology as a contour - tree skeleton and designing a graph neural network to model the surface topology at different spatial scales, the experimental results show that this model outperforms several baseline methods in terms of classification accuracy.

Deep Neural Network for 3D Surface Segmentation based on Contour Tree Hierarchy

Simultaneous Alignment and Surface Regression Using Hybrid 2D-3D Networks for 3D Coherent Layer Segmentation of Retinal OCT Images with Full and Annotations

Earth Imagery Segmentation on Terrain Surface with Limited Training Labels: A Semi-supervised Approach based on Physics-Guided Graph Co-Training

A Surface Graph Based Deep Learning Framework for Large-Scale Urban Mesh Semantic Segmentation

ISDNet: Integrating Shallow and Deep Networks for Efficient Ultra-high Resolution Segmentation

Deep Neural Networks for Surface Segmentation Meet Conditional Random Fields

DeepEdge: A multi-scale bifurcated deep network for top-down contour detection

Model-Informed Deep Learning for Surface Segmentation in Medical Imaging.

Hierarchical Insights: Exploiting Structural Similarities for Reliable 3D Semantic Segmentation

Deep Learning based 3D Segmentation: A Survey

Globally Optimal Segmentation of Mutually Interacting Surfaces using Deep Learning

Hierarchical Depthwise Graph Convolutional Neural Network for 3D Semantic Segmentation of Point Clouds

Globally Optimal Surface Segmentation using Deep Learning with Learnable Smoothness Priors

Deep Hierarchical Semantic Segmentation

A Novel Hierarchical Cross-Stream Aggregation Neural Network for Semantic Segmentation of 3-D Dental Surface Models

Open-set Hierarchical Semantic Segmentation for 3D Scene

A Texture Integrated Deep Neural Network for Semantic Segmentation of Urban Meshes

Human Segmentation With Deep Contour-Aware Network

Unsupervised 3D Shape Segmentation and Co-Segmentation Via Deep Learning.

Deep Learning-Based Segmentation of 3D Volumetric Image and Microstructural Analysis