Abstract:Overcoming catastrophic forgetting is a key difficulty for remote sensing image (RSI) classification in open-world applications. The core of this problem lies in the ability of RSI scene classification models to adapt to the changing environment and maintain the learned knowledge while continually learning new knowledge. Mainstream replay-based approaches overcome catastrophic forgetting by reenacting and retracing past experiences in the process of learning new data. However, such approaches rely heavily on the storage of historical data, and the recent rise of new paradigms based on prompt learning offers a new perspective of using only task-related "instructions" (i.e., prompts) to guide the model's continual learning and reasoning. Therein, the task knowledge encoded by the prompt improves the model's ability to overcome forgetting while reducing the amount of data and model parameters required by traditional data-driven approaches. Therefore, we propose a continual learning method based on prompt learning for RSI classification. We systematically analyze and reveal the potential of prompt learning for continual learning of RSI classification. Experiments on three publicly available remote sensing datasets show that prompt learning significantly outperforms two comparable methods on 3, 6, and 9 tasks, with an average accuracy (ACC) improvement of approximately 43%. Performance improvements of 4%–6% were achieved when compared with advanced prototype network methods. We found that prompt-generation strategies and prompt-related components significantly affect performance: (1) prompt-generation strategies are strongly correlated with the model's performance in overcoming catastrophic forgetting; (2) prompt-related components are correlated with RSIs of different scales. The new paradigm of prompt learning potentially provides a new idea for the continual learning problem of RSI classification.

Continual Panoptic Perception: Towards Multi-modal Incremental Interpretation of Remote Sensing Images

Panoptic Perception: A Novel Task and Fine-grained Dataset for Universal Remote Sensing Image Interpretation

Continual Contrastive Learning for Cross-Dataset Scene Classification

Continual Vision-Language Representation Learning with Off-Diagonal Information

Proper Reuse of Features Extractor for Real-time Continual Learning

Few-Shot MS and PAN Joint Classification with Improved Cross-Source Contrastive Learning

The CLEAR Benchmark: Continual LEArning on Real-World Imagery

Enhancing Visual Continual Learning with Language-Guided Supervision

Continual Learning by Using Information of Each Class Holistically.

Modeling Inner- and Cross-Task Contrastive Relations for Continual Image Classification

Self-Training and Curriculum Learning Guided Dynamic Refined Network for Remote Sensing Class-Incremental Semantic Segmentation

Spatial and Semantic Consistency Contrastive Learning for Self-Supervised Semantic Segmentation of Remote Sensing Images

ECLIPSE: Efficient Continual Learning in Panoptic Segmentation with Visual Prompt Tuning

Continual Learning for Remote Sensing Image Scene Classification With Prompt Learning

CTP: Towards Vision-Language Continual Pretraining Via Compatible Momentum Contrast and Topology Preservation

SegCLIP: Multimodal Visual-Language and Prompt Learning for High-Resolution Remote Sensing Semantic Segmentation

A Multi-Scale Progressive Collaborative Attention Network for Remote Sensing Fusion Classification.

CoLeCLIP: Open-Domain Continual Learning via Joint Task Prompt and Vocabulary Learning

CLRS: Continual Learning Benchmark for Remote Sensing Image Scene Classification

Cascadic Multireceptive Learning for Multispectral Pansharpening