Abstract:Deep neural networks have achieved promising progress in remote sensing (RS) image classification, for which the training process requires abundant samples for each class. However, it is time-consuming and unrealistic to annotate labels for each RS category, given the fact that the RS target database is increasing dynamically. Zero-shot learning (ZSL) allows for identifying novel classes that are not seen during training, which provides a promising solution for the aforementioned problem. However, previous ZSL models mainly depend on manually-labeled attributes or word embeddings extracted from language models to transfer knowledge from seen classes to novel classes. Besides, pioneer ZSL models use convolutional neural networks pre-trained on ImageNet, which focus on the main objects appearing in each image, neglecting the background context that also matters in RS scene classification. To address the above problems, we propose to collect visually detectable attributes automatically. We predict attributes for each class by depicting the semantic-visual similarity between attributes and images. In this way, the attribute annotation process is accomplished by machine instead of human as in other methods. Moreover, we propose a Deep Semantic-Visual Alignment (DSVA) that take advantage of the self-attention mechanism in the transformer to associate local image regions together, integrating the background context information for prediction. The DSVA model further utilizes the attribute attention maps to focus on the informative image regions that are essential for knowledge transfer in ZSL, and maps the visual images into attribute space to perform ZSL classification. With extensive experiments, we show that our model outperforms other state-of-the-art models by a large margin on a challenging large-scale RS scene classification benchmark.

A Multi-Group Multi-Stream attribute Attention network for fine-grained zero-shot learning

Multiscale Visual-Attribute Co-Attention for Zero-Shot Image Recognition

Multi-modal Generative Adversarial Network for Zero-Shot Learning

Stacked Semantic-Guided Attention Model for Fine-Grained Zero-Shot Learning.

Visual-guided attentive attributes embedding for zero-shot learning

Multi-Group Multi-Attention: Towards Discriminative Spatiotemporal Representation

Multi-Group Multi-Attention

Learning complementary semantic information for zero-shot recognition

Attentive Semantic Preservation Network for Zero-Shot Learning.

Multi-branch and Multi-scale Attention Learning for Fine-Grained Visual Categorization

PSVMA+: Exploring Multi-granularity Semantic-visual Adaption for Generalized Zero-shot Learning

Deep Semantic-Visual Alignment for Zero-Shot Remote Sensing Image Scene Classification

Adaptive multi-scale semantic fusion network for zero-shot learning

A Robust Generalized Zero-Shot Learning Method with Attribute Prototype and Discriminative Attention Mechanism

Attribute Attention for Semantic Disambiguation in Zero-Shot Learning

Attribute self-representation steered by exclusive lasso for zero-shot learning

Dual Relation Mining Network for Zero-Shot Learning

Multi-directional guidance network for fine-grained visual classification

Multi-Attention Multi-Class Constraint for Fine-grained Image Recognition

Dual Feature Augmentation Network for Generalized Zero-shot Learning

Multi-Attention Network For One Shot Learning