FERERO: A Flexible Framework for Preference-Guided Multi-Objective Learning

Lisha Chen,AFM Saif,Yanning Shen,Tianyi Chen

2024-12-03

Abstract:Finding specific preference-guided Pareto solutions that represent different trade-offs among multiple objectives is critical yet challenging in multi-objective problems. Existing methods are restrictive in preference definitions and/or their theoretical guarantees. In this work, we introduce a Flexible framEwork for pREfeRence-guided multi-Objective learning (FERERO) by casting it as a constrained vector optimization problem. Specifically, two types of preferences are incorporated into this formulation -- the relative preference defined by the partial ordering induced by a polyhedral cone, and the absolute preference defined by constraints that are linear functions of the objectives. To solve this problem, convergent algorithms are developed with both single-loop and stochastic variants. Notably, this is the first single-loop primal algorithm for constrained vector optimization to our knowledge. The proposed algorithms adaptively adjust to both constraint and objective values, eliminating the need to solve different subproblems at different stages of constraint satisfaction. Experiments on multiple benchmarks demonstrate the proposed method is very competitive in finding preference-guided optimal solutions. Code is available at <a class="link-external link-https" href="https://github.com/lisha-chen/FERERO/" rel="external noopener nofollow">this https URL</a>.

Machine Learning

What problem does this paper attempt to address?

This paper attempts to solve the problem of finding preference - guided solutions in multi - objective optimization problems, especially for the trade - offs between different objectives. Specifically, the paper proposes a new framework - FERERO (Flexible framEwork for pREfeRence - guided multi - Objective learning) to find Pareto solutions with specific preferences in multi - objective optimization. Existing methods have limitations in preference definition and theoretical guarantees, while FERERO solves these problems by introducing relative preferences and absolute preferences, and develops single - loop and stochastic variant algorithms with convergence.

### Problem Background

Multi - objective optimization problems involve multiple conflicting objectives, such as accuracy, fairness, and privacy. In practical applications, it is often necessary to find the optimal solutions that can reflect different trade - offs. Existing methods have limitations in preference definition and theoretical guarantees and cannot flexibly handle different preference requirements.

### Main Contributions of the FERERO Framework

1. **Flexibility**: The FERERO framework can capture more flexible preferences, including relative preferences (defined by the polyhedral cone induced by the partial order) and absolute preferences (defined by the linear function constraints of the objectives).
2. **Convergence**: Single - loop and stochastic variant algorithms with convergence are developed. This is the first single - loop master algorithm with convergence guarantees in constrained vector optimization.
3. **Adaptability**: The proposed algorithms can adaptively adjust constraints and objective values, eliminating the need to solve different sub - problems at different stages.
4. **Experimental Verification**: Experiments on multiple benchmark datasets show that the FERERO method is very competitive in finding preference - guided optimal solutions.

### Mathematical Formula Representation

- **Relative Preferences**: Defined by the polyhedral cone induced by the partial order:
\[
C=\{y\in\mathbb{R}^M\mid Ay\geq0\}
\]
where \(A\in\mathbb{R}^{M\times M}\) is a matrix.

- **Absolute Preferences**: Defined by the linear function constraints of the objectives:
\[
G(\theta)=B_gF(\theta)+b_g\leq0,\quad H(\theta)=B_hF(\theta)+b_h = 0
\]
where \(B_g\in\mathbb{R}^{M_g\times M}\), \(B_h\in\mathbb{R}^{M_h\times M}\), \(b_g\in\mathbb{R}^{M_g}\), \(b_h\in\mathbb{R}^{M_h}\).

- **Optimization Problem**:
\[
\min_{\theta\in\mathbb{R}^q}F(\theta)=[f_1(\theta),\ldots,f_M(\theta)]^{\top},\quad\text{s.t.}\quad G(\theta)\leq0,\quad H(\theta)=0
\]

### Summary

This paper solves the problem of finding preference - guided solutions in multi - objective optimization by introducing the FERERO framework, providing more flexible preference definitions and algorithms with convergence guarantees. This provides a new and effective tool for the field of multi - objective optimization, and has broad application prospects especially in fields such as machine learning and drug design.