Abstract:In this paper, we propose a simple yet efficient strategy for improving the multi-objective steepest descent method proposed by Fliege and Svaiter (Math Methods Oper Res, 2000, 3: 479--494). The core idea behind this strategy involves incorporating a positive modification parameter into the iterative formulation of the multi-objective steepest descent algorithm in a multiplicative manner. This modification parameter captures certain second-order information associated with the objective functions. We provide two distinct methods for calculating this modification parameter, leading to the development of two improved multi-objective steepest descent algorithms tailored for solving multi-objective optimization problems. Under reasonable assumptions, we demonstrate the convergence of sequences generated by the first algorithm toward a critical point. Moreover, for strongly convex multi-objective optimization problems, we establish the linear convergence to Pareto optimality of the sequence of generated points. The performance of the new algorithms is empirically evaluated through a computational comparison on a set of multi-objective test instances. The numerical results underscore that the proposed algorithms consistently outperform the original multi-objective steepest descent algorithm.

What problem does this paper attempt to address?

This paper mainly discusses the improvement strategies of the gradient descent method in multi-objective optimization problems. The existing multi-objective gradient descent algorithm (MSD) may not perform well in certain situations, possibly due to inappropriate step size selection. The authors propose a new strategy by introducing a positive correction parameter θk to modify the iteration formula in a multiplicative manner, which captures second-order information of the objective function around the point xk. They provide two different methods to calculate this correction parameter θk and construct two novel gradient descent algorithms for multi-objective optimization problems based on these improvements. The paper first reviews the gradient descent method in single-objective optimization problems and then turns to multi-objective optimization problems, where the objective functions are usually conflicting, so there is no unique solution to optimize all objectives simultaneously. The authors point out the problems existing in existing MSD algorithms, namely, poor convergence performance due to small step size. The paper then proposes two improved MSD algorithms that accumulate second-order information about the objective function using θk. The first method approximates the Hessian of each objective function with a common diagonal matrix, while the second method first obtains intermediate iterative points using the MSD algorithm, and then updates θk using the second-order information of these points and the original point. The paper proves that the first improved algorithm can converge to points that satisfy the necessary conditions for Pareto optimality under appropriate assumptions, and the linear convergence of the sequence is guaranteed for strongly convex multi-objective optimization problems. Numerical experiments demonstrate the superior performance of the proposed algorithm on multiple multi-objective optimization problems, showing better effectiveness compared to the original MSD algorithm and the multi-objective diagonal gradient descent algorithm in most cases. Overall, the goal of this paper is to improve the multi-objective gradient descent method and enhance its convergence and efficiency in solving multi-objective optimization problems by introducing a correction parameter.

Improvements to steepest descent method for multi-objective optimization

Convergence of Inexact Steepest Descent Algorithm for Multiobjective Optimizations on Riemannian Manifolds Without Curvature Constraints

Stochastic Steepest-Descent Optimization Of Multiple-Objective Mobile Sensor Coverage

A Barzilai-Borwein Descent Method for Multiobjective Optimization Problems

Extended Newton Methods for Multiobjective Optimization: Majorizing Function Technique and Convergence Analysis

Scaled Proximal Gradient Methods for Multiobjective Optimization: Improved Linear Convergence and Nesterov's Acceleration

An Improved Physical Programming Method for Multi-Objective Inverse Problems

Memory gradient method for multiobjective optimization

Descent Methods for Vector Optimization Problems: A Majorization-Minimization Perspective

Conjugate gradient methods without line search for multiobjective optimization

Vector Design Optimizations Using an Improved Cross-Entropy Method

Barzilai-Borwein Descent Methods for Multiobjective Optimization Problems with Variable Trade-off Metrics

Mirror descent method for stochastic multi-objective optimization

A global Barzilai and Borwein's gradient normalization descent method for multiobjective optimization

An Efficient Derivative-Free Algorithm for Bound Constrained Mixed-Integer Optimization

Two efficient gradient methods with approximately optimal stepsizes based on regularization models for unconstrained optimization

Multipliers Correction Methods for Optimization Problems over the Stiefel Manifold

Barzilai-Borwein Proximal Gradient Methods for Multiobjective Composite Optimization Problems with Improved Linear Convergence

A modified Polak-Ribiere-Polyak type conjugate gradient method with two stepsize strategies for vector optimization

Two Improved Algorithms Based on Nash-Pareto Strategy and Their Applications

An Improved Dai–Kou Conjugate Gradient Algorithm for Unconstrained Optimization