Abstract:Humans are able to recognize structured relations in observation, allowing us to decompose complex scenes into simpler parts and abstract the visual world at multiple levels. However, such hierarchical reasoning ability of human perception remains largely unexplored in current literature of semantic segmentation. Existing works are often aware of flatten labels and distinguish all the semantic categories exclusively for each pixel. In this work, we instead address hierarchical semantic segmentation (HSS), with the aim of providing a structured, pixel-wise description of visual observation in terms of a class hierarchy. We devise Hssn , a general HSS framework that tackles two critical issues in this task: i) how to efficiently adapt existing hierarchy-agnostic segmentation networks to the HSS setting, and ii) how to leverage the class hierarchy to regularize HSS network learning. To address i) , Hssn directly casts HSS as a pixel-wise multi-label classification task, only bringing minimal architecture change to current segmentation models. To solve ii) , Hssn first explores inherent properties of the hierarchy as a training objective, which enforces segmentation predictions to obey the hierarchy structure. Furthermore, with a set of hierarchy-induced margin constraints, Hssn efficiently reshapes the learned pixel embedding space, so as to generate hierarchy-aware pixel representations and facilitate structured segmentation eventually. Building upon Hssn , we further exploit the mutual exclusion relation between semantic labels and strengthen the margin based regularization strategy with more meaningful constrains, leading to Hssn +, a more effective framework for HSS. We conduct extensive experiments on six semantic segmentation datasets (i.e., Mapillary Vistas 2.0, Cityscapes, LIP, PASCAL-Person-Part, PASCAL-Part-58, and PASCAL-Part-108), with different class hierarchies, network architectures, and backbones, and the results confirm the generalization and superiority of our algorithms.

SegHSI: Semantic Segmentation of Hyperspectral Images With Limited Labeled Pixels

Advanced Global Prototypical Segmentation Framework for Few-Shot Hyperspectral Image Classification

Hyperspectral Image Super-Resolution Classification With A Small Training Set Using Spectral Variation Extended Endmember Library

Hyperspectral Image Instance Segmentation Using Spectral–Spatial Feature Pyramid Network

Semisupervised Classification Based on SLIC Segmentation for Hyperspectral Image

Weakly-supervised Semantic Segmentation in Cityscape Via Hyperspectral Image.

Superpixel-Guided Discriminative Low-Rank Representation of Hyperspectral Images for Classification

DSSNet: A Simple Dilated Semantic Segmentation Network for Hyperspectral Imagery Classification

Superpixel Segmentation for Hyperspectral Images Based on False Colour Composition with Colour Histogram Driving

Toward the Vectorization of Hyperspectral Imagery.

Hyperspectral Imagery Classification with Cascaded Support Vector Machines and Multi-Scale Superpixel Segmentation

Semantic Correntropy Representation for Hyperspectral Image Classification.

Unsupervised Hyperspectral Image Segmentation: Merging Spectral and Spatial Information in Boundary Adjustment

Few-Shot Hyperspectral Image Classification Based on Adaptive Subspaces and Feature Transformation

HSIGene: A Foundation Model For Hyperspectral Image Generation

Accurate Instance Segmentation for Remote Sensing Images via Adaptive and Dynamic Feature Learning

Semantic Hierarchy-Aware Segmentation

Semantic Segmentation of Hyperspectral Remote Sensing Images Based on PSE-UNet Model

Hyperspectral Image Classification Based on Semisupervised Self-Learning and Multiview Information Fusion

Efficient Semantic Segmentation of Hyperspectral Images Using Adaptable Rectangular Convolution