Abstract:To address the challenge of requiring a large amount of manually annotated data for semantic segmentation of remote sensing images using deep learning, a method based on self-supervised learning is proposed. Firstly, to simultaneously learn the global and local features of remote sensing images, a self-supervised learning network structure called TBSNet (Triple-Branch Self-supervised Network) is constructed. This network comprises an image transformation prediction branch, a global contrastive learning branch, and a local contrastive learning branch. The contrastive learning part of the network employs a novel data augmentation method to simulate positive pairs of the same remote sensing images under different weather conditions, enhancing the model's performance. Meanwhile, the model integrates channel attention and spatial attention mechanisms in the projection head structure of the global contrastive learning branch, and replaces a fully connected layer with a convolutional layer in the local contrastive learning branch, thus improving the model's feature extraction ability. Secondly, to mitigate the high computational cost during the pre-training phase, an algorithm optimization strategy is proposed using the TracIn method and sequential optimization theory, which increases the efficiency of pre-training. Lastly, by fine-tuning the model with a small amount of annotated data, effective semantic segmentation of remote sensing images is achieved even with limited annotated data. The experimental results indicate that with only 10% annotated data, the overall accuracy (OA) and recall of this model have improved by 4.60% and 4.88% respectively, compared to the traditional self-supervised model SimCLR (A Simple Framework for Contrastive Learning of Visual Representations). This provides significant application value for tasks such as semantic segmentation in remote sensing imagery and other computer vision domains.

SATS: Self-Attention Transfer for Continual Semantic Segmentation

SemiCVT: Semi-Supervised Convolutional Vision Transformer for Semantic Segmentation

Continual Attentive Fusion for Incremental Learning in Semantic Segmentation

Rethinking Self-Supervised Semantic Segmentation: Achieving End-to-End Segmentation

L2A: Learning Affinity from Attention for Weakly Supervised Continual Semantic Segmentation

Knowledge Transfer with Simulated Inter-Image Erasing for Weakly Supervised Semantic Segmentation

Semantic segmentation using cross-stage feature reweighting and efficient self-attention

Continual Learning for Image Segmentation with Dynamic Query

An Internal-External Constrained Distillation Framework for Continual Semantic Segmentation.

Fully Self-Supervised Learning for Semantic Segmentation

TCNet: Multiscale Fusion of Transformer and CNN for Semantic Segmentation of Remote Sensing Images

Efficient Semantic Segmentation Via Self-Attention and Self-Distillation

Transferable Semi-Supervised Semantic Segmentation

Learning Affinity from Attention: End-to-End Weakly-Supervised Semantic Segmentation with Transformers

Delving into Transformer for Incremental Semantic Segmentation

Hybrid Attention Fusion Embedded in Transformer for Remote Sensing Image Semantic Segmentation

TIKP: Text-to-Image Knowledge Preservation for Continual Semantic Segmentation

Self-Training and Curriculum Learning Guided Dynamic Refined Network for Remote Sensing Class-Incremental Semantic Segmentation

Self-Training for Class-Incremental Semantic Segmentation

Research on Semantic Segmentation Method of Remote Sensing Image Based on Self-supervised Learning

Mitigating Undisciplined Over-Smoothing in Transformer for Weakly Supervised Semantic Segmentation