Abstract:Low-rank adaptations (LoRA) are widely used to fine-tune large models across various domains for specific downstream tasks. While task-specific LoRAs are often available, concerns about data privacy and intellectual property can restrict access to training data, limiting the acquisition of a multi-task model through gradient-based training. In response, LoRA merging presents an effective solution by combining multiple LoRAs into a unified adapter while maintaining data privacy. Prior works on LoRA merging primarily frame it as an optimization problem, yet these approaches face several limitations, including the rough assumption about input features utilized in optimization, massive sample requirements, and the unbalanced optimization objective. These limitations can significantly degrade performance. To address these, we propose a novel optimization-based method, named IterIS: 1) We formulate LoRA merging as an advanced optimization problem to mitigate the rough assumption. Additionally, we employ an iterative inference-solving framework in our algorithm. It can progressively refine the optimization objective for improved performance. 2) We introduce an efficient regularization term to reduce the need for massive sample requirements (requiring only 1-5% of the unlabeled samples compared to prior methods). 3) We utilize adaptive weights in the optimization objective to mitigate potential unbalances in LoRA merging process. Our method demonstrates significant improvements over multiple baselines and state-of-the-art methods in composing tasks for text-to-image diffusion, vision-language models, and large language models. Furthermore, our layer-wise algorithm can achieve convergence with minimal steps, ensuring efficiency in both memory and computation.

Multi-LoRA Continual Learning Based Instruction Tuning Framework for Universal Information Extraction

InstructUIE: Multi-task Instruction Tuning for Unified Information Extraction.

A Framework to Implement 1+N Multi-task Fine-tuning Pattern in LLMs Using the CGC-LORA Algorithm

RUIE: Retrieval-based Unified Information Extraction using Large Language Model

MultiLoRA: Democratizing LoRA for Better Multi-Task Learning

MT4CrossOIE: Multi-stage Tuning for Cross-lingual Open Information Extraction

IterIS: Iterative Inference-Solving Alignment for LoRA Merging

Multimodal Instruction Tuning with Conditional Mixture of LoRA

Chain-of-LoRA: Enhancing the Instruction Fine-Tuning Performance of Low-Rank Adaptation on Diverse Instruction Set

Mixture-of-LoRAs: An Efficient Multitask Tuning for Large Language Models

UKT: A Unified Knowledgeable Tuning Framework for Chinese Information Extraction

Universal Information Extraction with Meta-Pretrained Self-Retrieval

MoDULA: Mixture of Domain-Specific and Universal LoRA for Multi-Task Learning

UMIE: Unified Multimodal Information Extraction with Instruction Tuning

InfLoRA: Interference-Free Low-Rank Adaptation for Continual Learning

Enhancing Task Performance in Continual Instruction Fine-tuning Through Format Uniformity

SuperLoRA: Parameter-Efficient Unified Adaptation of Multi-Layer Attention Modules

Visual Cue Enhancement and Dual Low-Rank Adaptation for Efficient Visual Instruction Fine-Tuning

LinguaLIFT: An Effective Two-stage Instruction Tuning Framework for Low-Resource Language Tasks

MTL-LoRA: Low-Rank Adaptation for Multi-Task Learning

LoraRetriever: Input-Aware LoRA Retrieval and Composition for Mixed Tasks in the Wild