Abstract:Federated learning has received increasing attention for its ability to collaborative learning without leaking privacy. Promising advances have been achieved under the assumption that participants share the same model structure. However, when participants independently customize their models, models suffer communication barriers, which leads the model heterogeneity problem. Moreover, in real scenarios, the data held by participants is often limited, making the local models trained only on private data present poor performance. Consequently, this paper studies a new challenging problem, namely few-shot model agnostic federated learning, where the local participants design their independent models from their limited private datasets. Considering the scarcity of the private data, we propose to utilize the abundant public available datasets for bridging the gap between local private participants. However, its usage also brings in two problems: inconsistent labels and large domain gap between the public and private datasets. To address these issues, this paper presents a novel framework with two main parts: 1) model agnostic federated learning, it performs public-private communication by unifying the model prediction outputs on the shared public datasets; 2) latent embedding adaptation, it addresses the domain gap with an adversarial learning scheme to discriminate the public and private domains. Together with theoretical generalization bound analysis, comprehensive experiments under various settings have verified our advantage over existing methods. It provides a simple but effective baseline for future advancement. The code is available at https://github.com/WenkeHuang/FSMAFL.

Towards Practical Few-shot Federated NLP

Federated Few-Shot Learning for Mobile NLP.

Ontology-enhanced Prompt-tuning for Few-shot Learning

PromptDA: Label-guided Data Augmentation for Prompt-based Few-shot Learners

Efficient Federated Learning for Modern NLP

SFPrompt: Communication-Efficient Split Federated Fine-Tuning for Large Pre-Trained Models over Resource-Limited Devices

Prompt-Based Self-training Framework for Few-Shot Named Entity Recognition.

FewFedWeight: Few-shot Federated Learning Framework Across Multiple NLP Tasks

Few-Shot Federated Learning: A Federated Learning Model for Small-Sample Scenarios

Making Pre-trained Language Models Better Learn Few-Shot Spoken Language Understanding in More Practical Scenarios.

FedPrompt: Communication-Efficient and Privacy Preserving Prompt Tuning in Federated Learning

Efficient Federated Prompt Tuning for Black-box Large Pre-trained Models

Adaptive Federated Few-Shot Feature Learning with Prototype Rectification

Enhancing Few-shot NER with Prompt Ordering based Data Augmentation

FsPONER: Few-shot Prompt Optimization for Named Entity Recognition in Domain-specific Scenarios

Prompt-Based Data Augmentation Framework for Few-Shot Named Entity Recognition

Few-Shot Model Agnostic Federated Learning

Tuning Language Models as Training Data Generators for Augmentation-Enhanced Few-Shot Learning

Federated Few-Shot Learning with Adversarial Learning

LM-CPPF: Paraphrasing-Guided Data Augmentation for Contrastive Prompt-Based Few-Shot Fine-Tuning