Abstract:Multi-task learning (MTL) benefits the fine-tuning of large language models (LLMs) by providing a single model with improved performance and generalization ability across tasks, presenting a resource-efficient alternative to developing separate models for each task. Yet, existing MTL strategies for LLMs often fall short by either being computationally intensive or failing to ensure simultaneous task convergence. This paper presents CoBa, a new MTL approach designed to effectively manage task convergence balance with minimal computational overhead. Utilizing Relative Convergence Scores (RCS), Absolute Convergence Scores (ACS), and a Divergence Factor (DF), CoBa dynamically adjusts task weights during the training process, ensuring that the validation loss of all tasks progress towards convergence at an even pace while mitigating the issue of individual task divergence. The results of our experiments involving three disparate datasets underscore that this approach not only fosters equilibrium in task convergence but enhances the LLMs' performance by up to 13% relative to the second-best baselines. Code is open-sourced at <a class="link-external link-https" href="https://github.com/codefuse-ai/MFTCoder" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

This paper attempts to solve two key problems in multi - task learning (MTL) fine - tuning of large - scale language models (LLMs): 1. **Minimizing additional computational cost**: Existing multi - task learning methods are often computationally expensive, especially when dealing with large - scale language models. This limits the practical applications of these methods, especially in resource - limited situations. 2. **Ensuring the simultaneous convergence of all tasks**: Current methods have difficulty ensuring that multiple tasks converge at the same rate during the training process, resulting in some tasks may converge prematurely or overfit, while other tasks may converge slowly or diverge. This imbalance will affect the overall performance of the model. To this end, the paper proposes CoBa (Convergence Balancer), a new multi - task learning method, which aims to balance the convergence speed between tasks by dynamically adjusting the loss weights of each task while maintaining low computational overhead. CoBa utilizes relative convergence scores (RCS), absolute convergence scores (ACS), and divergence factors (DF) to achieve this goal. Specifically: - **RCS**: It is used to evaluate the convergence speed of each task relative to other tasks. When the validation losses of all tasks are decreasing, RCS will reduce the weights of those tasks with faster convergence speeds and increase the weights of those tasks with slower convergence speeds. - **ACS**: It is used to evaluate the historical convergence rate of a single task. When a certain task starts to diverge, ACS will reduce the weight of this task, thereby preventing overfitting. - **DF**: It is used to monitor the divergence trend of tasks and determine the influence degree of RCS and ACS on the final weight. When all tasks are converging, DF makes RCS dominant; when a task starts to diverge, DF makes ACS dominant. Through these mechanisms, CoBa can not only balance the convergence speed between tasks but also improve the overall performance of the model. The experimental results show that CoBa significantly improves the effect of multi - task learning on four different datasets, and the performance is improved by up to 13% compared with the second - best baseline method.

CoBa: Convergence Balancer for Multitask Finetuning of Large Language Models

CoBa: Convergence Balancer for Multitask Finetuning of Large Language Models

MFTCoder: Boosting Code LLMs with Multitask Fine-Tuning

Cross-model Control: Improving Multiple Large Language Models in One-time Training

A Framework to Implement 1+N Multi-task Fine-tuning Pattern in LLMs Using the CGC-LORA Algorithm

Balancing Speciality and Versatility: a Coarse to Fine Framework for Supervised Fine-tuning Large Language Model

Cobra: Extending Mamba to Multi-Modal Large Language Model for Efficient Inference

Independent Component Alignment for Multi-Task Learning

Robust Estimator Based Adaptive Multi-Task Learning

CommonIT: Commonality-Aware Instruction Tuning for Large Language Models via Data Partitions

Multi-Task Cooperative Learning via Searching for Flat Minima

MTLComb: multi-task learning combining regression and classification tasks for joint feature selection

Towards Task-Conflicts Momentum-Calibrated Approach for Multi-task Learning

MoDULA: Mixture of Domain-Specific and Universal LoRA for Multi-Task Learning

CodeACT: Code Adaptive Compute-efficient Tuning Framework for Code LLMs

Delta-CoMe: Training-Free Delta-Compression with Mixed-Precision for Large Language Models

Dual-Balancing for Multi-Task Learning

Broad Multitask Learning System With Group Sparse Regularization

MoTCoder: Elevating Large Language Models with Modular of Thought for Challenging Programming Tasks

AdaTask: A Task-aware Adaptive Learning Rate Approach to Multi-task Learning