Abstract:The generalisation capacity of Multi-Task Learning (MTL) suffers when unrelated tasks negatively impact each other by updating shared parameters with conflicting gradients. This is known as negative transfer and leads to a drop in MTL accuracy compared to single-task learning (STL). Lately, there has been a growing focus on the fairness of MTL models, requiring the optimization of both accuracy and fairness for individual tasks. Analogously to negative transfer for accuracy, task-specific fairness considerations might adversely affect the fairness of other tasks when there is a conflict of fairness loss gradients between the jointly learned tasks - we refer to this as Bias Transfer. To address both negative- and bias-transfer in MTL, we propose a novel method called FairBranch, which branches the MTL model by assessing the similarity of learned parameters, thereby grouping related tasks to alleviate negative transfer. Moreover, it incorporates fairness loss gradient conflict correction between adjoining task-group branches to address bias transfer within these task groups. Our experiments on tabular and visual MTL problems show that FairBranch outperforms state-of-the-art MTLs on both fairness and accuracy.

What problem does this paper attempt to address?

This paper attempts to solve two main problems in multi - task learning (MTL): **negative transfer and bias transfer**. ### 1. Negative Transfer In traditional multi - task learning, when there are conflicting gradient updates between different tasks, the update of shared parameters may lead to a decline in the performance of some tasks. This phenomenon is known as **negative transfer**, that is, the performance of a multi - task model on a specific task is not as good as that of a single - task model. Specifically, when the optimization directions between tasks are inconsistent, the conflicting gradients of shared parameters will lead to a decline in model performance. ### 2. Bias Transfer In recent years, researchers have begun to pay attention to the fairness issue of multi - task learning models. Studies have shown that multi - task models may make biased predictions for specific demographic groups (such as gender, race, etc.). To optimize fairness, researchers have introduced fairness loss functions. However, when the fairness loss gradients between different tasks conflict, the fairness improvement of one task may have a negative impact on the fairness of other tasks, which is called **bias transfer**. ### Solution: FairBranch To solve the above problems, the author proposes a new method - **FairBranch**. This method alleviates negative transfer and bias transfer through the following two mechanisms: - **Task grouping based on parameter similarity**: FairBranch reduces the impact of negative transfer by evaluating the parameter similarity between tasks and grouping related tasks. Specifically, it creates branches according to the similarity of task parameters, so that tasks with similar parameters are grouped into the same group, thereby reducing the impact of conflicting gradients. - **Fairness gradient conflict correction**: Within each task group, FairBranch also performs fairness gradient conflict correction to solve the bias transfer problem. Specifically, it corrects the conflicts of fairness loss gradients within each branch to ensure that the fairness within the task group does not affect each other. ### Experimental Results The experimental results show that FairBranch is significantly superior to existing multi - task learning methods on multiple datasets, especially in dealing with negative transfer and bias transfer. Specifically: - **ACS - PUMS dataset**: On demographic data, FairBranch successfully reduces negative transfer and also performs well on fairness indicators. - **CelebA dataset**: On image data, FairBranch can also effectively alleviate negative transfer and bias transfer, especially when dealing with a large number of tasks. ### Summary This paper successfully solves the problems of negative transfer and bias transfer in multi - task learning by proposing the FairBranch method. By combining task grouping and fairness gradient conflict correction, FairBranch not only improves the accuracy of the model but also ensures the fairness of the model and is applicable to multiple types of datasets.

FairBranch: Mitigating Bias Transfer in Fair Multi-task Learning

Learning to Teach Fairness-aware Deep Multi-task Learning

Fair Resource Allocation in Multi-Task Learning

Understanding and Improving Fairness-Accuracy Trade-offs in Multi-Task Learning

Learning Boost by Exploiting the Auxiliary Task in Multi-task Domain

Towards Impartial Multi-task Learning.

Equitable Multi-task Learning

Taking Advantage of Multitask Learning for Fair Classification

ForkMerge: Mitigating Negative Transfer in Auxiliary-Task Learning

Fairness in Multi-Task Learning via Wasserstein Barycenters

Task Grouping for Automated Multi-Task Machine Learning via Task Affinity Prediction

Multi-Task Distillation: Towards Mitigating the Negative Transfer in Multi-Task Learning

A Model-Agnostic Approach to Mitigate Gradient Interference for Multi-Task Learning

FairTL: A Transfer Learning Approach for Bias Mitigation in Deep Generative Models

Multitask Learning Can Improve Worst-Group Outcomes

Transferring Fairness using Multi-Task Learning with Limited Demographic Information

Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data

Dual-Balancing for Multi-Task Learning

Task Uncertainty Loss Reduce Negative Transfer in Asymmetric Multi-task Feature Learning

Improving Multi-task Learning via Seeking Task-based Flat Regions

AdaTask: A Task-aware Adaptive Learning Rate Approach to Multi-task Learning