Abstract:To infer unknown remote sensing scenarios, most existing technologies use a supervised learning paradigm to train deep neural network (DNN) models on closed datasets. This paradigm faces challenges such as highly spatiotemporal variants and ever-changing scale-heterogeneous remote sensing scenarios. Additionally, DNN models cannot scale to new scenarios. Lifelong learning is an effective solution to these problems. Current lifelong learning approaches focus on overcoming the catastrophic forgetting issue (i.e., a successive increase in heterogeneous remote sensing scenes causes models to forget historical scenes) while ignoring the knowledge recall issue (i.e., how to facilitate the learning of new scenes by recalling historical experiences), which is a significant problem. This article proposes a lifelong learning framework called asymmetric collaborative network (SCN) for lifelong remote sensing image (RSI) classification. This framework consists of two structurally distinct networks: a preserving network (Pres-Net) and a transient network (Trans-Net), which imitate the long- and short-term memory processes in the brain, respectively. Moreover, this framework is based on two synergistic knowledge transfer mechanisms: triple distillation and prior feature fusion. The triple distillation mechanism enables knowledge persistence from Trans-Net to Pres-Net to achieve better memorization; the prior feature fusion mechanism enables knowledge transfer from Pres-Net to Trans-Net to achieve better recall. Experiments on three open datasets demonstrate the effectiveness of SCN for three-, six-, and nine-task-length learning. The idea of asymmetric separation networks and the synergistic strategy proposed in this article are expected to provide new solutions to the translatability of the classification of RSIs in real-world scenarios. The source codes are available at https://github.com/GeoX-Lab/SCN.

MiSSNet: Memory-Inspired Semantic Segmentation Augmentation Network for Class-Incremental Learning in Remote Sensing Images

Class-Incremental Learning for Semantic Segmentation in Aerial Imagery Via Distillation in All Aspects

A Multiscale Incremental Learning Network for Remote Sensing Scene Classification

Class-Incremental Semantic Segmentation of Aerial Images Via Pixel-Level Feature Generation and Task-Wise Distillation

Self-Training and Curriculum Learning Guided Dynamic Refined Network for Remote Sensing Class-Incremental Semantic Segmentation

SSUL: Semantic Segmentation with Unknown Label for Exemplar-based Class-Incremental Learning

Inherit With Distillation and Evolve With Contrast: Exploring Class Incremental Semantic Segmentation Without Exemplar Memory

DILRS: Domain-Incremental Learning for Semantic Segmentation in Multi-Source Remote Sensing Data

Semi-Supervised Semantic Segmentation of Remote Sensing Images With Iterative Contrastive Network

CIT: Rethinking Class-incremental Semantic Segmentation with a Class Independent Transformation

MIGN: Multiscale Image Generation Network for Remote Sensing Image Semantic Segmentation

Multi-scale Spatial Aggregation Network for Remote Sensing Image Segmentation

MMSMCNet: Modal Memory Sharing and Morphological Complementary Networks for RGB-T Urban Scene Semantic Segmentation

Saving 100x Storage: Prototype Replay for Reconstructing Training Sample Distribution in Class-Incremental Semantic Segmentation

Memory-Based Contrastive Learning with Optimized Sampling for Incremental Few-Shot Semantic Segmentation

Endpoints Weight Fusion for Class Incremental Semantic Segmentation

Towards Realistic Incremental Scenario in Class Incremental Semantic Segmentation

EndpointsWeight Fusion for Class Incremental Semantic Segmentation

Better Memorization, Better Recall: A Lifelong Learning Framework for Remote Sensing Image Scene Classification

Causes of Catastrophic Forgetting in Class-Incremental Semantic Segmentation

Interaction semantic segmentation network via progressive supervised learning