Abstract:We propose a preference-learning algorithm for uncovering Decision Makers’ (DMs’) contingent evaluation strategies in the context of multiple criteria sorting. We assume the preference information in the form of holistic assignment examples derived from the analysis of alternatives’ performance vectors and textual descriptions. We characterize the decision policies using a mixture of threshold-based, value-driven preference models and associated latent topics. The latter serve as the stimuli underlying the contingency in decision behavior. Such a probabilistic model is constructed by using a flexible and nonparametric Bayesian framework. The proposed method adopts a hierarchical Dirichlet process as the prior so that a group of DMs can share a countably infinite number of contingent models and topics. For all DMs, it automatically identifies the components representing their evaluation strategies adequately. The posterior is summarized by using the Hamiltonian Monte Carlo sampling method. We demonstrate the method’s practical usefulness in a real-world recruitment problem considered by a Chinese IT company. We also compare the approach with counterparts that use a single preference model, implement the parametric framework, or consider each DM’s preferences individually. The results indicate that our approach performs favorably in both interpreting DMs’ contingent decision behavior and recommending decisions on new alternatives. Furthermore, the approach’s performance and robustness are investigated through a computational experiment involving real-world data sets. History: Accepted by Ram Ramesh, Area Editor for Data Science and Machine Learning. Funding: J. Liu received financial support from the National Natural Science Foundation of China [Grants 72071155 and 71701160]. M. Kadziński received financial support from the Polish National Science Center under the SONATA BIS project [Grant DEC-2019/34/E/HS4/00045]. X. Liao received financial support from the National Natural Science Foundation of China [Grant 71872144]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2023.1292 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2021.0328 ) at ( http://dx.doi.org/10.5281/zenodo.7608750 ).

Integrating Machine Learning Models to Learn Potentially Non-Monotonic Preferences for Multi-Criteria Sorting from Large-Scale Assignment Examples

An incremental preference elicitation-based approach to learning potentially non-monotonic preferences in multi-criteria sorting

Lexicographic optimization-based approaches to learning a representative model for multi-criteria sorting with non-monotonic criteria

Data-driven preference learning methods for value-driven multiple criteria sorting with interacting criteria

Data-driven Preference Learning Methods for Sorting Problems with Multiple Temporal Criteria

Deep preference learning for multiple criteria decision analysis

Learning MR-Sort Models from Non-Monotone Data

Modeling Contingent Decision Behavior: A Bayesian Nonparametric Preference-Learning Approach

Preference learning and multiple criteria decision aiding: differences, commonalities, and synergies–part I

Monotone-Value Neural Networks: Exploiting Preference Monotonicity in Combinatorial Assignment

A hybrid machine learning framework for analyzing human decision making through learning preferences

Robust ordinal regression in preference learning and ranking

Selection of a representative sorting model in a preference disaggregation setting: a review of existing procedures, new proposals, and experimental comparison

Multicriteria Machine Learning Model Assessment—Residuum Analysis Review

Hybrid Multi-Criteria Preference Ranking by Subsorting

Dynamic Assortment with Online Learning under Threshold Multinomial Logit Model

Ordinal Preference Optimization: Aligning Human Preferences via NDCG

Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization

Preference Estimation under Bounded Rationality: Identification of Attribute Non-Attendance in Stated-Choice Data Using a Support Vector Machines Approach

Learning preference representations based on Choquet integrals for multicriteria decision making

Multi-Criteria Comparison as a Method of Advancing Knowledge-Guided Machine Learning