Abstract:Sentinel-2 images provide a rich source of information for a variety of land cover, vegetation, and environmental monitoring applications due to their high spectral, spatial, and temporal resolutions. Recently, deep learning-based classification of Sentinel-2 time series becomes a popular solution to vegetation classification and land cover mapping, but it often demands a large number of manually annotated labels. Improving classification performance with limited labeled data is still a challenge in many real-world remote sensing applications. To address label scarcity, we present SITS-Former (SITS stands for Satellite Image Time Series and Former stands for Transformer), a pre-trained representation model for Sentinel-2 time series classification. SITS-Former adopts a Transformer encoder as the backbone and takes time series of image patches as input to learn spatio-spectral-temporal features. According to the principles of self-supervised learning, we pre-train SITS-Former on massive unlabeled Sentinel-2 time series via a missing-data imputation proxy task. Given an incomplete time series with some patches being masked randomly, the network is asked to regress the central pixels of these masked patches based on the residual ones. By doing so, the network can capture high-level spatial and temporal dependencies from the data to learn discriminative features. After pre-training, the network can adapt the learned features to a target classification task through fine-tuning. As far as we know, this is the first study that exploits self-supervised learning for patch-based representation learning and classification of SITS. We quantitatively evaluate the quality of the learned features by transferring them on two crop classification tasks, showing that SITS-Former outperforms state-of-the-art approaches and yields a significant improvement (2.64%∼3.30% in overall accuracy) over the purely supervised model. The proposed model provides an effective tool for SITS-related applications as it greatly reduces the burden of manual labeling. The source code will be released at https://github.com/linlei1214/SITS-Former upon publication.

S4Former: A Spectral–Spatial Sparse Selection Transformer for Multispectral Remote Sensing Scene Classification

CSwT-SR: Conv-Swin Transformer for Blind Remote Sensing Image Super-Resolution with Amplitude-Phase Learning and Structural Detail Alternating Learning

A Spatial–Spectral Transformer for Hyperspectral Image Classification Based on Global Dependencies of Multi-Scale Features

SVAFormer: Integrating Random and Hierarchical Spectral View Attention for Hyperspectral Image Classification

MFormer: Taming Masked Transformer for Unsupervised Spectral Reconstruction

Vision Transformer with Sparse Scan Prior

TTST: A Top-k Token Selective Transformer for Remote Sensing Image Super-Resolution

MASSFormer: Memory-Augmented Spectral-Spatial Transformer for Hyperspectral Image Classification

SCViT: A Spatial-Channel Feature Preserving Vision Transformer for Remote Sensing Image Scene Classification

Spectral Superresolution Using Transformer with Convolutional Spectral Self-Attention

MMFormer: Multimodal Transformer Using Multiscale Self-Attention for Remote Sensing Image Classification

Cross-Spatial Pixel Integration and Cross-Stage Feature Fusion Based Transformer Network for Remote Sensing Image Super-Resolution

Coarse-to-Fine Sparse Transformer for Hyperspectral Image Reconstruction

SITS-Former: A pre-trained spatio-spectral-temporal representation model for Sentinel-2 time series classification

Selective Transformer for Hyperspectral Image Classification

Enhanced Window-Based Self-Attention with Global and Multi-Scale Representations for Remote Sensing Image Super-Resolution

MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction

3D-Convolution Guided Spectral-Spatial Transformer for Hyperspectral Image Classification

Weighted residual self-attention graph-based transformer for spectral–spatial hyperspectral image classification

Supervised-unsupervised combined transformer for spectral compressive imaging reconstruction

CS2DT: Cross Spatial–Spectral Dense Transformer for Hyperspectral Image Classification