Abstract:Sub-pixel mapping is the prevailing approach for dealing with the mixed pixel effect in urban land use/land cover classification, by reconstructing the sub-pixel-scale distribution inside each mixed-pixel based on spatial autocorrelation. However, 1) traditional spatial autocorrelation is limited to a local window, which cannot model the teleconnection between two locations or objects that are far apart and 2) autocorrelation is based on the idea of “the more proximate, the more similar”, which relies on a distance-weight decay parameter and cannot characterize the rich variety of mutual information in spatially heterogenous areas in urban. In this research, we develop and demonstrate a learnable correlation-based sub-pixel mapping (LECOS) method. 1) We use the “mutual retrieval” mechanism of the self-attention operation to model teleconnections that enable more distant locations or objects to be mutually correlated and 2) we design a parameter-free “self-attention in self-attention” operation to learn adaptively the diverse global correlation patterns between pixel and sub-pixel. The learned spatial correlations are then used for reasoning the sub-pixel-scale distribution of each class. We validated our method on the most challenging public datasets of urban scenes, which exhibit considerable spatial heterogeneity with complex structures and broken objects. The learned building-tree, building-road and road-tree correlation patterns contributed most to the sub-pixel reconstruction result of the urban scenes, consistent with in-situ reference data. We further explored the model's explicability in a large-area of several metropolises in China, by mapping land cover in these cities at a 2 m very fine spatial resolution using 10 m Sentinel-2 input images, and found that the derived result not only revealed rich urban spatial heterogeneity, but also that the learned correlation was indicative of urban pattern dynamics, suggesting the potential for greater understanding of issues such as urban fairness, accessibility, human exposure and sustainability.

Land-Use Scene Classification in High-Resolution Remote Sensing Images Using Improved Correlatons

Exploring the Synergistic Use of Multi-Scale Image Object Metrics for Land-Use/Land-Cover Mapping Using an Object-Based Approach

A multi-scale dense residual correlation network for remote sensing scene classification

Improved Robust Kernel Subspace for Object-Based Registration and Change Detection

Effective and Efficient Midlevel Visual Elements-Oriented Land-Use Classification Using VHR Remote Sensing Images

A new strategy based on multi-source remote sensing data for improving the accuracy of land use/cover change classification

Bag-of-visual-words and Spatial Extensions for Land-Use Classification

Assisted learning for land use classification: The important role of semantic correlation between heterogeneous images

Emmcnn: An Etps-Based Multi-Scale And Multi-Feature Method Using Cnn For High Spatial Resolution Image Land-Cover Classification

Robust Land Cover Classification with Local-Global Information Decoupling to Address Remote Sensing Anomalous Data

An Object-Based Classification Approach For High-Spatial Resolution Imagery

SPLC: An Attention-Based End-to-End Superpixels and Land Cover Classification Jointly Learning Architecture for HR Remote Sensing Image

Long-Range Correlation Supervision for Land-Cover Classification from Remote Sensing Images

How Well Do Deep Learning-Based Methods for Land Cover Classification and Object Detection Perform on High Resolution Remote Sensing Imagery?

Spatial context-aware method for urban land use classification using street view images

MHFNet: an Improved HGR Multimodal Network for Informative Correlation Fusion in Remote Sensing Image Classification

Super-Resolution Mapping Based on Spatial–Spectral Correlation for Spectral Imagery

Very fine spatial resolution urban land cover mapping using an explicable sub-pixel mapping network based on learnable spatial correlation

Semi-supervised segmentation of land cover images using nonlinear canonical correlation analysis with multiple features and t-SNE

Classification of Very-High-Spatial-Resolution Aerial Images Based on Multiscale Features with Limited Semantic Information

Integrating Multi-Temporal Spectral and Structural Information from ALOS/AVNIR-2 Images to Map Heterogeneous Land Use/Cover: A Hybrid Approach