When Multi-Task Learning Meets Partial Supervision: A Computer Vision Review

Maxime Fontana,Michael Spratling,Miaojing Shi

2024-08-28

Abstract:Multi-Task Learning (MTL) aims to learn multiple tasks simultaneously while exploiting their mutual relationships. By using shared resources to simultaneously calculate multiple outputs, this learning paradigm has the potential to have lower memory requirements and inference times compared to the traditional approach of using separate methods for each task. Previous work in MTL has mainly focused on fully-supervised methods, as task relationships can not only be leveraged to lower the level of data-dependency of those methods but they can also improve performance. However, MTL introduces a set of challenges due to a complex optimisation scheme and a higher labeling requirement. This review focuses on how MTL could be utilised under different partial supervision settings to address these challenges. First, this review analyses how MTL traditionally uses different parameter sharing techniques to transfer knowledge in between tasks. Second, it presents the different challenges arising from such a multi-objective optimisation scheme. Third, it introduces how task groupings can be achieved by analysing task relationships. Fourth, it focuses on how partially supervised methods applied to MTL can tackle the aforementioned challenges. Lastly, this review presents the available datasets, tools and benchmarking results of such methods.

Machine Learning,Artificial Intelligence,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

This paper aims to explore the application of multi-task learning (MTL) in a partially supervised setting and how it overcomes the challenges encountered in traditional multi-task learning. Specifically:

1. **Multi-task Parameter Sharing Techniques**: First, it analyzes traditional multi-task parameter sharing methods and discusses different strategies for feature fusion and knowledge decomposition.

2. **Optimization Challenges**: It introduces the issues that arise during multi-objective optimization, including techniques for loss functions, gradient-based methods, and other task balancing techniques.

3. **Task Grouping**: By analyzing the relationships between tasks, it determines which tasks can benefit from each other and groups them accordingly.

4. **Multi-task Learning under Partial Supervision**: It focuses on how to use partially supervised methods to address the aforementioned challenges and improve the effectiveness of multi-task learning.

Additionally, the paper covers related datasets, tools, and benchmark results, and reviews existing multi-task learning literature, particularly its applications in the field of computer vision. Through this research, the authors hope to provide a comprehensive understanding framework for the future development of multi-task learning, especially regarding the impact of increasing the number and diversity of tasks.