Abstract:There are a lot of real-world black-box optimization problems that need to optimize multiple criteria simultaneously. However, in a multi-objective optimization (MOO) problem, identifying the whole Pareto front requires the prohibitive search cost, while in many practical scenarios, the decision maker (DM) only needs a specific solution among the set of the Pareto optimal solutions. We propose a Bayesian optimization (BO) approach to identifying the most preferred solution in the MOO with expensive objective functions, in which a Bayesian preference model of the DM is adaptively estimated by an interactive manner based on the two types of supervisions called the pairwise preference and improvement request. To explore the most preferred solution, we define an acquisition function in which the uncertainty both in the objective functions and the DM preference is incorporated. Further, to minimize the interaction cost with the DM, we also propose an active learning strategy for the preference estimation. We empirically demonstrate the effectiveness of our proposed method through the benchmark function optimization and the hyper-parameter optimization problems for machine learning models.

What problem does this paper attempt to address?

This paper presents a multi-objective optimization method based on Bayesian optimization, combined with active learning to acquire the preference information of decision makers (DM). In multi-objective optimization problems, it is usually necessary to optimize multiple objective functions simultaneously, and finding the entire Pareto front requires high search costs. However, in practical scenarios, the decision maker may only need a specific solution from the set of Pareto optimal solutions. For this purpose, the paper proposes a method to estimate the decision maker's preference model interactively, using two types of weak supervision: pairwise comparison and improvement request. The main contributions of the paper are as follows: 1. Proposing a multi-objective optimization algorithm that combines Bayesian optimization and preference learning to optimize both the objective functions and the decision maker's preferences using Bayesian modeling. 2. Introducing a Bayesian preference learning method based on pairwise comparison and improvement request, where improvement request is a novel preference learning paradigm. 3. Presenting an active learning strategy for selecting effective queries (pairwise comparisons or improvement requests) to reduce the interaction cost with the decision maker. 4. Introducing a more flexible utility function based on Gaussian processes to adapt to the decision maker's preferences. 5. Demonstrating the superiority of the proposed method over baseline methods through experiments on benchmark function optimization and hyperparameter optimization of machine learning models. In the paper, both the objective functions and the decision maker's utility function are assumed to be unknown black-box functions. The objective functions are modeled using Gaussian processes (GP), while for the utility function, the paper primarily considers the Chebyshev scalarization function and proposes a variant that combines GP and monotonicity information to improve flexibility. In addition, to estimate the utility function, the paper utilizes the decision maker's pairwise comparisons and improvement requests to acquire weak supervision information and updates the Bayesian model based on this information. Finally, the effectiveness of the proposed method is verified through numerical experiments.

Multi-Objective Bayesian Optimization with Active Preference Learning

Bayesian Optimization For Multi-Objective Mixed-Variable Problems

Preference-Optimized Pareto Set Learning for Blackbox Optimization

Bayesian Optimization for Multi-objective Optimization and Multi-point Search

Pareto Front-Diverse Batch Multi-Objective Bayesian Optimization

Multi-agent Black-box Optimization using a Bayesian Approach to Alternating Direction Method of Multipliers

Preference-Aware Constrained Multi-Objective Bayesian Optimization

Non-Myopic Multi-Objective Bayesian Optimization

A Bayesian Optimization Framework for Finding Local Optima in Expensive Multi-Modal Functions

Direct Preference-Based Evolutionary Multi-Objective Optimization with Dueling Bandit

Dynamic Multi-objective Ensemble of Acquisition Functions in Batch Bayesian Optimization

Human-in-the-Loop Policy Optimization for Preference-Based Multi-Objective Reinforcement Learning

A Bi-Objective Optimization Based Acquisition Strategy for Batch Bayesian Global Optimization

Multi-Step Budgeted Bayesian Optimization with Unknown Evaluation Costs

R-MBO: A Multi-surrogate Approach for Preference Incorporation in Multi-objective Bayesian Optimisation

srMO-BO-3GP: A Sequential Regularized Multi-Objective Constrained Bayesian Optimization for Design Applications

High-Dimensional Multi-Objective Bayesian Optimization With Block Coordinate Updates: Case Studies in Intelligent Transportation System

Bayesian multi-objective optimization for stochastic simulators: an extension of the Pareto Active Learning method

Approximating Pareto Frontier Through Bayesian-optimization-directed Robust Multi-objective Reinforcement Learning

Expensive Multi-Objective Bayesian Optimization Based on Diffusion Models

Preferential Multi-Objective Bayesian Optimization