Abstract:Geometric information in the normalized digital surface models (nDSM) is highly correlated with the semantic class of the land cover. Exploiting two modalities (RGB and nDSM (height)) jointly has great potential to improve the segmentation performance. However, it is still an under-explored field in remote sensing due to the following challenges. First, the scales of existing datasets are relatively small and the diversity of existing datasets is limited, which restricts the ability of validation. Second, there is a lack of unified benchmarks for performance assessment, which leads to difficulties in comparing the effectiveness of different models. Last, sophisticated multi-modal semantic segmentation methods have not been deeply explored for remote sensing data. To cope with these challenges, in this paper, we introduce a new remote-sensing benchmark dataset for multi-modal semantic segmentation based on RGB-Height (RGB-H) data. Towards a fair and comprehensive analysis of existing methods, the proposed benchmark consists of 1) a large-scale dataset including co-registered RGB and nDSM pairs and pixel-wise semantic labels; 2) a comprehensive evaluation and analysis of existing multi-modal fusion strategies for both convolutional and Transformer-based networks on remote sensing data. Furthermore, we propose a novel and effective Transformer-based intermediary multi-modal fusion (TIMF) module to improve the semantic segmentation performance through adaptive token-level multi-modal <a class="link-external link-http" href="http://fusion.The" rel="external noopener nofollow">this http URL</a> designed benchmark can foster future research on developing new methods for multi-modal learning on remote sensing data. Extensive analyses of those methods are conducted and valuable insights are provided through the experimental results. Code for the benchmark and baselines can be accessed at \url{<a class="link-external link-https" href="https://github.com/EarthNets/RSI-MMSegmentation" rel="external noopener nofollow">this https URL</a>}.

Technical Report on A Generic Multi-Dimensional Data Generator for Earth Mover ’ s Distance Similarity Analysis

A Progressive Approach for Computing the Earth Mover's Distance

Efficient EMD-based Similarity Search Via Batch Pruning and Incremental Computation (extended Abstract)

EarthMapper: A Tool Box for the Semantic Segmentation of Remote Sensing Imagery

EMDS-5: Environmental Microorganism image dataset Fifth Version for multiple image analysis tasks

MELODY-JOIN: Efficient Earth Mover's Distance Similarity Joins Using MapReduce

Efficient approximation of Earth Mover's Distance Based on Nearest Neighbor Search

Towards End-to-End Embroidery Style Generation: A Paired Dataset and Benchmark

Two novel benchmark datasets from ArcGIS and bing world imagery for remote sensing image retrieval

EMDS-6: Environmental Microorganism Image Dataset Sixth Version for Image Denoising, Segmentation, Feature Extraction, Classification and Detection Methods Evaluation

EMDS-6: Environmental Microorganism Image Dataset Sixth Version for Image Denoising, Segmentation, Feature Extraction, Classification, and Detection Method Evaluation

Extraction and Comparison of Gene Expression Patterns from 2d Rna in Situ Hybridization Images

The earth mover's distance as a semantic measure for document similarity.

MMM-RS: A Multi-modal, Multi-GSD, Multi-scene Remote Sensing Dataset and Benchmark for Text-to-Image Generation

GAMUS: A Geometry-aware Multi-modal Semantic Segmentation Benchmark for Remote Sensing Data

The Generation and Application of Medical Image Grid Based on Differential Geometric Features.

BDGS: A Scalable Big Data Generator Suite in Big Data Benchmarking

A novel document similarity measure based on earth mover's distance

Heads-Join: Efficient Earth Mover's Distance Similarity Joins on Hadoop

EM-stellar: benchmarking deep learning for electron microscopy image segmentation

Clustering in Dynamic Environments: A Framework for Benchmark Dataset Generation With Heterogeneous Changes