Abstract:Remote sensing image scene classification takes image blocks as classification units and predicts their semantic descriptors. Because it is difficult to obtain enough labeled samples for all classes of remote sensing image scenes, zero-shot classification methods which can recognize image scenes that are not seen in the training stage are of great significance. By projecting the image visual features and the class semantic features into the latent space and ensuring their alignment, the variational autoencoder (VAE) generative model has been applied to address remote-sensing image scene classification under a zero-shot setting. However, the VAE model takes the element-wise square error as the reconstruction loss, which may not be suitable for measuring the reconstruction quality of the visual and semantic features. Therefore, this paper proposes to augment the VAE models with the generative adversarial network (GAN) to make use of the GAN's discriminator in order to learn a suitable reconstruction quality metric for VAE. To promote feature alignment in the latent space, we have also proposed cross-modal feature-matching loss to make sure that the visual features of one class are aligned with the semantic features of the class and not those of other classes. Based on a public dataset, our experiments have shown the effects of the proposed improvements. Moreover, taking the ResNet models of ResNet18, extracting 512-dimensional visual features, and ResNet50 and ResNet101, both extracting 2048-dimensional visual features for testing, the impact of the different visual feature extractors has also been investigated. The experimental results show that better performance is achieved by ResNet18. This indicates that more layers of the extractors and larger dimensions of the extracted features may not contribute to the image scene classification under a zero-shot setting.

GAN-MVAE: A discriminative latent feature generation framework for generalized zero-shot learning

GENERATING MANIFOLD-ALIGNED SEMANTIC FEATURE FOR ZERO-SHOT LEARNING

Zero-Shot Learning with Generative Latent Prototype Model.

Zero-VAE-GAN: Generating Unseen Features for Generalized and Transductive Zero-Shot Learning

Learning discriminative and representative feature with cascade GAN for generalized zero-shot learning

Dual VAEGAN: A generative model for generalized zero-shot learning

Contrastive Visual Feature Filtering for Generalized Zero-Shot Learning

Generalized Zero-Shot Learning With Multiple Graph Adaptive Generative Networks

Integrating Adversarial Generative Network with Variational Autoencoders Towards Cross-Modal Alignment for Zero-Shot Remote Sensing Image Scene Classification

Generalized Zero-Shot Learning using Multimodal Variational Auto-Encoder with Semantic Concepts

A Discriminative Cross-Aligned Variational Autoencoder for Zero-Shot Learning

Beyond Normal Distribution: More Factual Feature Generation Network for Generalized Zero-Shot Learning

VAE-GAN3D: Leveraging image-based semantics for 3D zero-shot recognition

Multi-modal Generative Adversarial Network for Zero-Shot Learning

Towards Discriminative Feature Generation for Generalized Zero-Shot Learning

Generative-based hybrid model with semantic representations for generalized zero-shot learning

Model Selection for Generalized Zero-shot Learning

CVAE-GAN: Fine-Grained Image Generation through Asymmetric Training

Leveraging Dual Variational Autoencoders and Generative Adversarial Networks for Enhanced Multimodal Interaction in Zero-Shot Learning

A Universal Discriminator for Zero-Shot Generalization

Modality independent adversarial network for generalized zero shot image classification