Abstract: Zero-shot learning is a learning regime that recognizes unseen classes by generalizing the visual-semantic relationship learned from the seen classes. To obtain an effective ZSL model, one may resort to curating training samples from multiple sources, which may inevitably raise the privacy concerns about data sharing across different organizations. In this paper, we propose a novel Federated Zero-Shot Learning FedZSL framework, which learns a central model from the decentralized data residing on edge devices. To better generalize to previously unseen classes, FedZSL allows the training data on each device sampled from the non-overlapping classes, which are far from the i.i.d. that traditional federated learning commonly assumes. We identify two key challenges in our FedZSL protocol: 1) the trained models are prone to be biased to the locally observed classes, thus failing to generalize to the unseen classes and/or seen classes appeared on other devices; 2) as each category in the training data comes from a single source, the central model is highly vulnerable to model replacement (backdoor) attacks. To address these issues, we propose three local objectives for visual-semantic alignment and cross-device alignment through relation distillation, which leverages the normalized class-wise covariance to regularize the consistency of the prediction logits across devices. To defend against the backdoor attacks, a feature magnitude defending technique is proposed. As malicious samples are less correlated to the given semantic attributes, the visual features of low magnitude will be discarded to stabilize model updates. The effectiveness and robustness of FedZSL are demonstrated by extensive experiments conducted on three zero-shot benchmark datasets.

Embarrassingly Easy Zero-Shot Image Recognition

Zero-Shot Detection with Transferable Object Proposal Mechanism.

Joint Learning of Attended Zero-Shot Features and Visual-Semantic Mapping.

Transductive Unbiased Embedding for Zero-Shot Learning

Zero-Shot Embedding via Regularization-Based Recollection and Residual Familiarity Processes

Meta-Transfer Networks for Zero-Shot Learning

Learning discriminative visual semantic embedding for zero-shot recognition

Zero-Shot Leaning With Manifold Embedding

Zero-Shot Learning on Semantic Class Prototype Graph

OntoZSL: Ontology-enhanced Zero-shot Learning

Zero-Shot Learning via Category-Specific Visual-Semantic Mapping and Label Refinement

Manifold Embedding for Zero-Shot Recognition

Manifold Regularized Cross-Modal Embedding for Zero-Shot Learning

Zero-Shot Learning with Transferred Samples

Learn More from Less: Generalized Zero-Shot Learning with Severely Limited Labeled Data

Towards Effective Deep Embedding for Zero-Shot Learning

Federated Zero-Shot Learning for Visual Recognition

Learning object-centric complementary features for zero-shot learning

Zero-Shot Recognition Using Dual Visual-Semantic Mapping Paths.

Zero-Shot Learning posed as a Missing Data Problem

Towards Zero-Shot Learning: A Brief Review and an Attention-Based Embedding Network