Abstract:While "instruction-tuned" generative large language models (LLMs) have demonstrated an impressive ability to generalize to new tasks, the training phases heavily rely on large amounts of diverse and high-quality instruction data (such as ChatGPT and GPT-4). Unfortunately, acquiring high-quality data, especially when it comes to human-written data, can pose significant challenges both in terms of cost and accessibility. Moreover, concerns related to privacy can further limit access to such data, making the process of obtaining it a complex and nuanced undertaking. Consequently, this hinders the generality of the tuned models and may restrict their effectiveness in certain contexts. To tackle this issue, our study introduces a new approach called Federated Instruction Tuning (FedIT), which leverages federated learning (FL) as the learning framework for the instruction tuning of LLMs. This marks the first exploration of FL-based instruction tuning for LLMs. This is especially important since text data is predominantly generated by end users. Therefore, it is imperative to design and adapt FL approaches to effectively leverage these users' diverse instructions stored on local devices, while preserving privacy and ensuring data security. In the current paper, by conducting widely used GPT-4 auto-evaluation, we demonstrate that by exploiting the heterogeneous and diverse sets of instructions on the client's end with the proposed framework FedIT, we improved the performance of LLMs compared to centralized training with only limited local instructions. Further, in this paper, we developed a Github repository named Shepherd. This repository offers a foundational framework for exploring federated fine-tuning of LLMs using heterogeneous instructions across diverse categories.

FewFedPIT: Towards Privacy-preserving and Few-shot Federated Instruction Tuning

FedPSE: Personalized Sparsification with Element-wise Aggregation for Federated Learning

Federated Instruction Tuning of LLMs with Domain Coverage Augmentation

Towards Building the Federated GPT: Federated Instruction Tuning

Federated Data-Efficient Instruction Tuning for Large Language Models

FedPT: Federated Proxy-Tuning of Large Language Models on Resource-Constrained Edge Devices

Data Quality Control in Federated Instruction-tuning of Large Language Models

Leveraging Unstructured Text Data for Federated Instruction Tuning of Large Language Models

When Federated Learning Meets Pre-trained Language Models' Parameter-Efficient Tuning Methods

Personalized Federated Few-Shot Learning

Efficient Federated Learning with Enhanced Privacy via Lottery Ticket Pruning in Edge Computing

Federated Prediction-Powered Inference from Decentralized Data

LF3PFL: A Practical Privacy-Preserving Federated Learning Algorithm Based on Local Federalization Scheme

Automated Federated Pipeline for Parameter-Efficient Fine-Tuning of Large Language Models

Privacy-Preserving Federated Learning via Dataset Distillation

PPFed: A Privacy-Preserving and Personalized Federated Learning Framework

FedPFT: Federated Proxy Fine-Tuning of Foundation Models

A TEE-Based Federated Privacy Protection Method: Proposal and Implementation

Enhancing Privacy in Federated Learning through Local Training

FedBPT: Efficient Federated Black-box Prompt Tuning for Large Language Models

FedFed: Feature Distillation Against Data Heterogeneity in Federated Learning