Abstract:Semantic segmentation of remote sensing images plays a vital role in a wide range of Earth Observation applications, such as land-use land-cover (LULC) mapping, environment monitoring, and sustainable development. Driven by rapid developments in artificial intelligence, deep learning (DL) has emerged as the mainstream for semantic segmentation and has achieved many breakthroughs in the field of remote sensing. However, most DL-based methods focus on unimodal visual data while ignoring rich multimodal information involved in the real world. Nonvisual data, such as text, can gather extra knowledge from the real world, which can strengthen the interpretability, reliability, and generalization of visual models. Inspired by this, we propose a novel metadata-collaborative segmentation network (MetaSegNet) that applies vision-language representation learning for the semantic segmentation of remote sensing images. Unlike the common model structure that only uses unimodal visual data, we extract the key characteristic (e.g., the climate zone) from freely available remote sensing image metadata and transfer it into geographic text prompts via the generic ChatGPT. Then, we construct an image encoder, a text encoder, and a crossmodal attention fusion subnetwork to extract the image and text feature and apply image-text interaction. Benefiting from such a design, the proposed MetaSegNet not only demonstrates superior generalization in zero-shot testing but also achieves competitive accuracy with the state-of-the-art semantic segmentation methods on the large-scale OpenEarthMap dataset [70.4% mean intersection over union (mIoU)] and the Potsdam dataset (93.3% mean score) as well as the LoveDA dataset (52.0% mIoU).

MetaSeg: A Survey of Meta-Learning for Image Segmentation

Meta-seg: A Survey of Meta-Learning for Image Segmentation

Deep Learning for Visual Segmentation: A Review

A Survey of Meta-learning for Classification Tasks

Meta-Seg: A Generalized Meta-Learning Framework for Multi-Class Few-Shot Semantic Segmentation.

MetaSegNet: Metadata-Collaborative Vision-Language Representation Learning for Semantic Segmentation of Remote Sensing Images

Image Segmentation Using Deep Learning: A Survey

Meta-Learners for Few-Shot Weakly-Supervised Medical Image Segmentation

MetaSeg: Content-Aware Meta-Net for Omni-Supervised Semantic Segmentation

Meta-Learning 3D Shape Segmentation Functions

A Survey on Image Semantic Segmentation Using Deep Learning Techniques

Weakly Supervised Few-Shot Segmentation Via Meta-Learning

Predicting Scores of Medical Imaging Segmentation Methods with Meta-Learning

Meta-learning approaches for learning-to-learn in deep learning: A survey

A Survey on Label-efficient Deep Image Segmentation: Bridging the Gap between Weak Supervision and Dense Prediction

Semi-supervised Meta-learning with Disentanglement for Domain-generalised Medical Image Segmentation

Unsupervised Meta-learning of Figure-Ground Segmentation via Imitating Visual Effects

A Survey on Deep Learning Technique for Video Segmentation