Abstract:Multi-task learning (MTL) aims to empower a model to tackle multiple tasks simultaneously. A recent development known as task arithmetic has revealed that several models, each fine-tuned for distinct tasks, can be directly merged into a single model to execute MTL without necessitating a retraining process using the initial training data. Nevertheless, this direct addition of models often leads to a significant deterioration in the overall performance of the merged model. This decline occurs due to potential conflicts and intricate correlations among the multiple tasks. Consequently, the challenge emerges of how to merge pre-trained models more effectively without using their original training data. This paper introduces an innovative technique called Adaptive Model Merging (AdaMerging). This approach aims to autonomously learn the coefficients for model merging, either in a task-wise or layer-wise manner, without relying on the original training data. Specifically, our AdaMerging method operates as an automatic, unsupervised task arithmetic scheme. It leverages entropy minimization on unlabeled test samples from the multi-task setup as a surrogate objective function to iteratively refine the merging coefficients of the multiple models. Our experimental findings across eight tasks demonstrate the efficacy of the AdaMerging scheme we put forth. Compared to the current state-of-the-art task arithmetic merging scheme, AdaMerging showcases a remarkable 11\% improvement in performance. Notably, AdaMerging also exhibits superior generalization capabilities when applied to unseen downstream tasks. Furthermore, it displays a significantly enhanced robustness to data distribution shifts that may occur during the testing phase.

Adaptive Adversarial Multi-task Representation Learning.

A Deep Multi-Task Representation Learning Method for Time Series Classification and Retrieval.

Robust Estimator Based Adaptive Multi-Task Learning

Efficient Multi-Task Reinforcement Learning via Task-Specific Action Correction

AdaMerging: Adaptive Model Merging for Multi-Task Learning

AdaTask: A Task-aware Adaptive Learning Rate Approach to Multi-task Learning

Task Adaptive Multi-learner Network for Joint CTR and CVR Estimation.

Boosting Exploration in Multi-Task Reinforcement Learning using Adversarial Networks

Theoretical Study of Conflict-Avoidant Multi-Objective Reinforcement Learning

Adaptive Activation Network and Functional Regularization for Efficient and Flexible Deep Multi-Task Learning

Improved Active Multi-Task Representation Learning via Lasso

Towards Impartial Multi-task Learning.

A Brain-Inspired Incremental Multi-task Reinforcement Learning Approach

Bridging Multi-Task Learning and Meta-Learning: Towards Efficient Training and Effective Adaptation

Task Variance Regularized Multi-Task Learning.

Equitable Multi-task Learning

Learning from Similar Linear Representations: Adaptivity, Minimaxity, and Robustness

Multi-Task Models Adversarial Attacks

A Decentralized Policy Gradient Approach to Multi-task Reinforcement Learning

Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data

Multi-Task Reinforcement Learning with Mixture of Orthogonal Experts