Abstract:Finetuning large language models (LLMs) in federated learning (FL) settings has become increasingly important as it allows resource-constrained devices to finetune a model using private data. However, finetuning LLMs using backpropagation requires excessive memory (especially from intermediate activations) for resource-constrained devices. While Forward-mode Auto-Differentiation (AD) can significantly reduce memory footprint from activations, we observe that directly applying it to LLM finetuning results in slow convergence and poor accuracy. In this paper, we introduce Spry, an FL algorithm that splits trainable weights of an LLM among participating clients, such that each client computes gradients using forward-mode AD that are closer estimations of the true gradients. Spry achieves a low memory footprint, high accuracy, and fast convergence. We formally prove that the global gradients in Spry are unbiased estimators of true global gradients for homogeneous data distributions across clients, while heterogeneity increases bias of the estimates. We also derive Spry's convergence rate, showing that the gradients decrease inversely proportional to the number of FL rounds, indicating the convergence up to the limits of heterogeneity. Empirically, Spry reduces the memory footprint during training by 1.4-7.1x in contrast to backpropagation, while reaching comparable accuracy, across a wide range of language tasks, models, and FL settings. Spry reduces the convergence time by 1.2-20.3x and achieves 5.2-13.5% higher accuracy against zero-order methods. When finetuning Llama2-7B with LoRA, compared to the peak memory consumption of 33.9GB of backpropagation, Spry only consumes 6.2GB of peak memory. For OPT13B, the reduction is from 76.5GB to 10.8GB. Spry makes feasible previously impossible FL deployments on commodity edge devices. Our source code is available at <a class="link-external link-https" href="https://github.com/Astuary/Spry" rel="external noopener nofollow">this https URL</a>.

FFSplit: Split Feed-Forward Network For Optimizing Accuracy-Efficiency Trade-off in Language Model Inference

Efficient and Economic Large Language Model Inference with Attention Offloading

Task Scheduling for Efficient Inference of Large Language Models on Single Moderate GPU Systems

Hardware-Guided Symbiotic Training for Compact, Accurate, yet Execution-Efficient LSTM

FoldGPT: Simple and Effective Large Language Model Compression Scheme

DP-FFN: Block-Based Dynamic Pooling for Accelerating Feed-Forward Layers in Transformers

Building on Efficient Foundations: Effectively Training LLMs with Structured Feedforward Layers

FinGPT-HPC: Efficient Pretraining and Finetuning Large Language Models for Financial Applications with High-Performance Computing

Communication Compression for Tensor Parallel LLM Inference

Variator: Accelerating Pre-trained Models with Plug-and-Play Compression Modules

Efficient Transformer-based Large Scale Language Representations using Hardware-friendly Block Structured Pruning

Large Language Models Compression via Low-Rank Feature Distillation

Activation Sparsity Opportunities for Compressing General Large Language Models

FlightLLM: Efficient Large Language Model Inference with a Complete Mapping Flow on FPGAs

An Efficient 2D Method for Training Super-Large Deep Learning Models

Enhancing Inference Efficiency of Large Language Models: Investigating Optimization Strategies and Architectural Innovations

FPGA Acceleration of Recurrent Neural Network Based Language Model

GroupReduce: Block-Wise Low-Rank Approximation for Neural Language Model Shrinking

Thinking Forward: Memory-Efficient Federated Finetuning of Language Models

Efficient LLM inference solution on Intel GPU

Accelerating Large Language Model Training with Hybrid GPU-based Compression