Abstract:Generative models (e.g., generative adversarial network (GAN)) have advanced zero-shot learning (ZSL). Studies on the generative ZSL methods typically produce visual features of unseen classes to mitigate the issue of lacking unseen samples based on the predefined class semantic prototypes. As these empirically designed prototypes are not able to faithfully represent the actual semantic prototypes of visual features (i.e., visual prototypes), existing methods limit their ability to synthesize visual features that accurately represent real features and prototypes. We formulate this phenomenon as a visual-semantic domain shift problem. It prevents the generative models from further improving the ZSL performance. In this paper, we propose a dynamic semantic prototype learning (DSP) method to align the empirical and actual semantic prototypes for synthesizing accurate visual features. The alignment is conducted by jointly refining semantic prototypes and visual features so that the generator synthesizes visual features which are close to the real ones. We utilize a visual$\rightarrow$semantic mapping network (V2SM) to map both the synthesized and real features into the class semantic space. The V2SM benefits the generator to synthesize visual representations with rich semantics. The real/synthesized visual features supervise our visual-oriented semantic prototype evolving network (VOPE) where the predefined class semantic prototypes are iteratively evolved to become dynamic semantic prototypes. Such prototypes are then fed back to the generative network as conditional supervision. Finally, we enhance visual features by fusing the evolved semantic prototypes into their corresponding visual features. Our extensive experiments on three benchmark datasets show that our DSP improves existing generative ZSL methods, \textit{e.g.}, the average improvements of the harmonic mean over four baselines (e.g., CLSWGAN, f-VAEGAN, TF-VAEGAN and FREE) by 8.5\%, 8.0\% and 9.7\% on CUB, SUN and AWA2, respectively.

DSP: Dynamic Semantic Prototype for Generative Zero-Shot Learning

Evolving Semantic Prototype Improves Generative Zero-Shot Learning

GENERATING MANIFOLD-ALIGNED SEMANTIC FEATURE FOR ZERO-SHOT LEARNING

Zero-Shot Learning with Generative Latent Prototype Model.

Visual-Augmented Dynamic Semantic Prototype for Generative Zero-Shot Learning

Attentive Semantic Preservation Network for Zero-Shot Learning.

Multi-modal Generative Adversarial Network for Zero-Shot Learning

Visual feature synthesis with semantic reconstructor for traditional and generalized zero‐shot object classification

Visual-Semantic Aligned Bidirectional Network for Zero-Shot Learning

Dual Adversarial Semantics-Consistent Network for Generalized Zero-Shot Learning

Progressive Semantic-Visual Mutual Adaption for Generalized Zero-Shot Learning

Learning Modality-Consistent Latent Representations for Generalized Zero-Shot Learning

Residual-Prototype Generating Network for Generalized Zero-Shot Learning

Semantic Compression Embedding for Generative Zero-Shot Learning

Dual-stream Generative Adversarial Networks for Distributionally Robust Zero-Shot Learning.

Bidirectional Generative Transductive Zero-Shot Learning.

Visual Data Synthesis Via GAN for Zero-Shot Video Classification

Progressive Semantic-Guided Vision Transformer for Zero-Shot Learning

Generative-based hybrid model with semantic representations for generalized zero-shot learning

SR-GAN: Semantic Rectifying Generative Adversarial Network for Zero-shot Learning