DLoRA: Distributed Parameter-Efficient Fine-Tuning Solution for Large Language Model

Chao Gao,Sai Qian Zhang

2024-04-08

Abstract:To enhance the performance of large language models (LLM) on downstream tasks, one solution is to fine-tune certain LLM parameters and make it better align with the characteristics of the training dataset. This process is commonly known as parameter-efficient fine-tuning (PEFT). Due to the scale of LLM, PEFT operations are usually executed in the public environment (e.g., cloud server). This necessitates the sharing of sensitive user data across public environments, thereby raising potential privacy concerns. To tackle these challenges, we propose a distributed PEFT framework called DLoRA. DLoRA enables scalable PEFT operations to be performed collaboratively between the cloud and user devices. Coupled with the proposed Kill and Revive algorithm, the evaluation results demonstrate that DLoRA can significantly reduce the computation and communication workload over the user devices while achieving superior accuracy and privacy protection.

Machine Learning,Artificial Intelligence,Computation and Language,Distributed, Parallel, and Cluster Computing

What problem does this paper attempt to address?

### The Problem This Paper Aims to Solve

This paper aims to address the issues of privacy and computational resource allocation during the Parameter-Efficient Fine-Tuning (PEFT) of large language models (LLMs) for downstream tasks.

Specifically:

1. **Privacy Protection**:
- In traditional PEFT methods, user data needs to be uploaded to the cloud for fine-tuning, which may lead to privacy leakage risks.
- To protect user privacy, the paper proposes a distributed PEFT framework called DLoRA, which allows PEFT operations to be collaboratively completed between cloud servers and user devices, ensuring that sensitive data does not leave the user's device.

2. **Computational Resource Allocation**:
- Solely relying on either the cloud or edge devices for LLM fine-tuning has limitations: cloud-based solutions may raise privacy concerns, while edge devices, although they can protect privacy, have limited computational resources.
- DLoRA alleviates the burden on the cloud by offloading part of the computational load to user devices, thereby enhancing system scalability.

3. **Reduction of Computational and Communication Overhead**:
- The introduction of the Kill and Revive algorithm dynamically identifies the subset of LLM parameters most sensitive to training data, significantly reducing the computational and communication burden on user devices.
- Experiments have shown that this algorithm can maintain or even improve model performance while reducing computational costs.

In summary, the core objective of the paper is to develop a distributed PEFT solution that both protects user privacy and efficiently utilizes computational resources.