Differentially Private Low-Rank Adaptation of Large Language Model Using Federated Learning

Xiao-Yang Liu,Rongyi Zhu,Daochen Zha,Jiechao Gao,Shan Zhong,Matt White,Meikang Qiu

2024-06-02

Abstract:The surge in interest and application of large language models (LLMs) has sparked a drive to fine-tune these models to suit specific applications, such as finance and medical science. However, concerns regarding data privacy have emerged, especially when multiple stakeholders aim to collaboratively enhance LLMs using sensitive data. In this scenario, federated learning becomes a natural choice, allowing decentralized fine-tuning without exposing raw data to central servers. Motivated by this, we investigate how data privacy can be ensured in LLM fine-tuning through practical federated learning approaches, enabling secure contributions from multiple parties to enhance LLMs. Yet, challenges arise: 1) despite avoiding raw data exposure, there is a risk of inferring sensitive information from model outputs, and 2) federated learning for LLMs incurs notable communication overhead. To address these challenges, this article introduces DP-LoRA, a novel federated learning algorithm tailored for LLMs. DP-LoRA preserves data privacy by employing a Gaussian mechanism that adds noise in weight updates, maintaining individual data privacy while facilitating collaborative model training. Moreover, DP-LoRA optimizes communication efficiency via low-rank adaptation, minimizing the transmission of updated weights during distributed training. The experimental results across medical, financial, and general datasets using various LLMs demonstrate that DP-LoRA effectively ensures strict privacy constraints while minimizing communication overhead.

Machine Learning,Cryptography and Security

What problem does this paper attempt to address?

The paper aims to address the issue of data privacy when applying large language models (LLMs) in specific domains such as finance and healthcare, and proposes a practical federated learning method to achieve data privacy protection. Specifically, the paper addresses the following aspects:

1. **Data Privacy Protection**: How to ensure data privacy is not compromised when fine-tuning LLMs using data from multiple institutions. Existing federated learning methods can avoid exposing raw data to a central server, but there is still a risk of inferring sensitive information from the model outputs.

2. **Communication Overhead**: Directly applying federated learning to LLMs results in significant communication overhead due to the frequent transmission of complex model updates.

To address the above issues, the authors propose the DP-LoRA algorithm, a federated learning algorithm that combines differential privacy mechanisms and low-rank adaptation techniques. DP-LoRA ensures individual data privacy by adding Gaussian noise to weight updates, while reducing communication overhead through low-rank adaptation. Experimental results show that DP-LoRA can effectively protect privacy and reduce communication costs across various datasets, including medical, financial, and general datasets.