Abstract:In order to create machine learning systems that serve a variety of users well, it is vital to not only achieve high average performance but also ensure equitable outcomes across diverse groups. However, most machine learning methods are designed to improve a model's average performance on a chosen end task without consideration for their impact on worst group error. Multitask learning (MTL) is one such widely used technique. In this paper, we seek not only to understand the impact of MTL on worst-group accuracy but also to explore its potential as a tool to address the challenge of group-wise fairness. We primarily consider the standard setting of fine-tuning a pre-trained model, where, following recent work \citep{gururangan2020don, dery2023aang}, we multitask the end task with the pre-training objective constructed from the end task data itself. In settings with few or no group annotations, we find that multitasking often, but not consistently, achieves better worst-group accuracy than Just-Train-Twice (JTT; \citet{pmlr-v139-liu21f}) -- a representative distributionally robust optimization (DRO) method. Leveraging insights from synthetic data experiments, we propose to modify standard MTL by regularizing the joint multitask representation space. We run a large number of fine-tuning experiments across computer vision and natural language processing datasets and find that our regularized MTL approach \emph{consistently} outperforms JTT on both average and worst-group outcomes. Our official code can be found here: \href{<a class="link-external link-https" href="https://github.com/atharvajk98/MTL-group-robustness.git" rel="external noopener nofollow">this https URL</a>}{\url{<a class="link-external link-https" href="https://github.com/atharvajk98/MTL-group-robustness" rel="external noopener nofollow">this https URL</a>}}.

FairBranch: Mitigating Bias Transfer in Fair Multi-task Learning

Learning to Teach Fairness-aware Deep Multi-task Learning

Fair Resource Allocation in Multi-Task Learning

Understanding and Improving Fairness-Accuracy Trade-offs in Multi-Task Learning

Learning Boost by Exploiting the Auxiliary Task in Multi-task Domain

Towards Impartial Multi-task Learning.

Equitable Multi-task Learning

Taking Advantage of Multitask Learning for Fair Classification

ForkMerge: Mitigating Negative Transfer in Auxiliary-Task Learning

Task-Feature Collaborative Learning with Application to Personalized Attribute Prediction

Lateral Transfer Learning for Multiagent Reinforcement Learning.

Fairness in Multi-Task Learning via Wasserstein Barycenters

Task Grouping for Automated Multi-Task Machine Learning via Task Affinity Prediction

Multi-Task Distillation: Towards Mitigating the Negative Transfer in Multi-Task Learning

A Model-Agnostic Approach to Mitigate Gradient Interference for Multi-Task Learning

FairTL: A Transfer Learning Approach for Bias Mitigation in Deep Generative Models

Improving Gradient Trade-offs between Tasks in Multi-task Text Classification

Multitask Learning Can Improve Worst-Group Outcomes

Transferring Fairness using Multi-Task Learning with Limited Demographic Information

Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data

Dual-Balancing for Multi-Task Learning