Abstract:With the rise of neural networks in various domains, multi-task learning (MTL) gained significant relevance. A key challenge in MTL is balancing individual task losses during neural network training to improve performance and efficiency through knowledge sharing across tasks. To address these challenges, we propose a novel task-weighting method by building on the most prevalent approach of Uncertainty Weighting and computing analytically optimal uncertainty-based weights, normalized by a softmax function with tunable temperature. Our approach yields comparable results to the combinatorially prohibitive, brute-force approach of Scalarization while offering a more cost-effective yet high-performing alternative. We conduct an extensive benchmark on various datasets and architectures. Our method consistently outperforms six other common weighting methods. Furthermore, we report noteworthy experimental findings for the practical application of MTL. For example, larger networks diminish the influence of weighting methods, and tuning the weight decay has a low impact compared to the learning rate.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper aims to solve a key challenge in multi - task learning (MTL): how to balance the losses of each task during the neural network training process to improve performance and efficiency. Specifically, the author proposes a new method based on uncertainty weighting, called Soft Optimal Uncertainty Weighting (UW - SO), to address the following problems: 1. **Imbalanced task losses**: - Different tasks may use different loss functions (such as L1 loss and cross - entropy loss), resulting in different loss scales. - Some tasks are more difficult than others and require more resources. - Even when using the same loss function, data noise and prediction uncertainty will also lead to different loss magnitudes among tasks. 2. **Limitations of existing methods**: - The equal weighting method (EW) may cause some tasks to dominate the training process. - Dynamic weighting methods (such as Uncertainty Weighting, UW) are vulnerable to poor initialization and overfitting. - The brute - force search method (such as Scalarization), although having good performance, has a too - high computational cost and is difficult to apply in practical scenarios. ### Solutions To solve the above problems, the author proposes the following solutions: 1. **Soft Optimal Uncertainty Weighting (UW - SO)**: - Based on the Uncertainty Weighting (UW) method, the optimal uncertainty weighting value is analytically derived and normalized by the softmax function with a temperature parameter. - This method only needs to adjust one hyperparameter (the temperature parameter \( T \)) and can achieve results comparable to or better than Scalarization with a significant reduction in computational steps. 2. **Extensive experimental verification**: - A large number of experiments were carried out on multiple datasets (such as NYUv2, Cityscapes, and CelebA) and different architectures to verify the effectiveness of the UW - SO method. - The experimental results show that UW - SO performs excellently in various tasks and network structures, especially in large - scale networks, where its performance is better than other common weighting methods. ### Main contributions - Proposed a new weighting method UW - SO based on uncertainty, which solves the initialization and overfitting problems of existing methods. - Verified the effectiveness and superiority of UW - SO through extensive experiments. - Provided some important observations in MTL practice, such as larger networks will reduce the performance differences between weighting methods, and the importance of learning rate adjustment. Through these contributions, the author hopes to provide a more efficient and easy - to - apply task - weighting method for the field of multi - task learning.

Analytical Uncertainty-Based Loss Weighting in Multi-Task Learning

Multi-Objective Optimization for Sparse Deep Multi-Task Learning

Task Uncertainty Loss Reduce Negative Transfer in Asymmetric Multi-task Feature Learning

MetaWeighting: Learning to Weight Tasks in Multi-Task Learning.

Robust Multi-Task Learning with Excess Risks

HydaLearn: Highly Dynamic Task Weighting for Multi-task Learning with Auxiliary Tasks

Towards Impartial Multi-task Learning.

Robust Analysis of Multi-Task Learning Efficiency: New Benchmarks on Light-Weighed Backbones and Effective Measurement of Multi-Task Learning Challenges by Feature Disentanglement

Dual-Balancing for Multi-Task Learning

Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics

The Multi-Task Learning with an Application of Pareto Improvement.

No More Tuning: Prioritized Multi-Task Learning with Lagrangian Differential Multiplier Methods

Sample-level weighting for multi-task learning with auxiliary tasks

Multiple Task Learning Using Iteratively Reweighted Least Square.

Multi-Adaptive Optimization for multi-task learning with deep neural networks

Equitable Multi-task Learning

Learning Boost by Exploiting the Auxiliary Task in Multi-task Domain

Task Weighting through Gradient Projection for Multitask Learning

A Model-Agnostic Approach to Mitigate Gradient Interference for Multi-Task Learning

Robust Estimator Based Adaptive Multi-Task Learning

AdaTask: A Task-aware Adaptive Learning Rate Approach to Multi-task Learning