Abstract:The goal of multi-task learning is to enable more efficient learning than single task learning by sharing model structures for a diverse set of tasks. A standard multi-task learning objective is to minimize the average loss across all tasks. While straightforward, using this objective often results in much worse final performance for each task than learning them independently. A major challenge in optimizing a multi-task model is the conflicting gradients, where gradients of different task objectives are not well aligned so that following the average gradient direction can be detrimental to specific tasks' performance. Previous work has proposed several heuristics to manipulate the task gradients for mitigating this problem. But most of them lack convergence guarantee and/or could converge to any Pareto-stationary point. In this paper, we introduce Conflict-Averse Gradient descent (CAGrad) which minimizes the average loss function, while leveraging the worst local improvement of individual tasks to regularize the algorithm trajectory. CAGrad balances the objectives automatically and still provably converges to a minimum over the average loss. It includes the regular gradient descent (GD) and the multiple gradient descent algorithm (MGDA) in the multi-objective optimization (MOO) literature as special cases. On a series of challenging multi-task supervised learning and reinforcement learning tasks, CAGrad achieves improved performance over prior state-of-the-art multi-objective gradient manipulation methods.

Adaptive multi-gradient methods for quasiconvex vector optimization and applications to multi-task learning

Self-adaptive algorithms for quasiconvex programming and applications to machine learning

Conjugate gradient methods without line search for multiobjective optimization

Adaptive generalized conditional gradient method for multiobjective optimization

Augmented Distributed Gradient Methods for Multi-Agent Optimization under Uncoordinated Constant Stepsizes

A Multi-task Learning Approach by Combining Derivative-Free and Gradient Methods.

Extended Newton Methods for Multiobjective Optimization: Majorizing Function Technique and Convergence Analysis

A modified subgradient extragradient method with non-monotonic step sizes for solving quasimonotone variational inequalities

Conflict-Averse Gradient Descent for Multi-task Learning

An Adaptive Stochastic Gradient Method with Non-negative Gauss-Newton Stepsizes

Gradient method with multiple damping for large-scale unconstrained optimization

Convergence Analysis of a Projected Gradient Method for Multiobjective Optimization Problems

On the Convergence of Adaptive Gradient Methods for Nonconvex Optimization

Nonmonotone Gradient Methods for Vector Optimization with a Portfolio Optimization Application

A three-term conjugate gradient descent method with some applications

A nonmonotone proximal quasi-Newton method for multiobjective optimization

A class of new three-term descent conjugate gradient algorithms for large-scale unconstrained optimization and applications to image restoration problems

Multiple Greedy Quasi-Newton Methods for Saddle Point Problems

A gradient descent akin method for constrained optimization: algorithms and applications

A Proximal Gradient Method with an Explicit Line search for Multiobjective Optimization

Stagewise Accelerated Stochastic Gradient Methods for Nonconvex Optimization