Abstract:We propose a preference-learning algorithm for uncovering Decision Makers’ (DMs’) contingent evaluation strategies in the context of multiple criteria sorting. We assume the preference information in the form of holistic assignment examples derived from the analysis of alternatives’ performance vectors and textual descriptions. We characterize the decision policies using a mixture of threshold-based, value-driven preference models and associated latent topics. The latter serve as the stimuli underlying the contingency in decision behavior. Such a probabilistic model is constructed by using a flexible and nonparametric Bayesian framework. The proposed method adopts a hierarchical Dirichlet process as the prior so that a group of DMs can share a countably infinite number of contingent models and topics. For all DMs, it automatically identifies the components representing their evaluation strategies adequately. The posterior is summarized by using the Hamiltonian Monte Carlo sampling method. We demonstrate the method’s practical usefulness in a real-world recruitment problem considered by a Chinese IT company. We also compare the approach with counterparts that use a single preference model, implement the parametric framework, or consider each DM’s preferences individually. The results indicate that our approach performs favorably in both interpreting DMs’ contingent decision behavior and recommending decisions on new alternatives. Furthermore, the approach’s performance and robustness are investigated through a computational experiment involving real-world data sets. History: Accepted by Ram Ramesh, Area Editor for Data Science and Machine Learning. Funding: J. Liu received financial support from the National Natural Science Foundation of China [Grants 72071155 and 71701160]. M. Kadziński received financial support from the Polish National Science Center under the SONATA BIS project [Grant DEC-2019/34/E/HS4/00045]. X. Liao received financial support from the National Natural Science Foundation of China [Grant 71872144]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2023.1292 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2021.0328 ) at ( http://dx.doi.org/10.5281/zenodo.7608750 ).

Heuristic-Based Weak Learning for Automated Decision-Making

Interactive Weak Supervision: Learning Useful Heuristics for Data Labeling

Specialization in Hierarchical Learning Systems

Leveraging Expert Consistency to Improve Algorithmic Decision Support

Human Reliance on Machine Learning Models When Performance Feedback is Limited: Heuristics and Risks

A Bandit Model for Human-Machine Decision Making with Private Information and Opacity

Leveraging automatic strategy discovery to teach people how to select better projects

Online Decision Mediation

Debiasing Machine Learning Models by Using Weakly Supervised Learning

Learning To Guide Human Decision Makers With Vision-Language Models

Learning Personalized Decision Support Policies

Algorithmic Decision Making in the Presence of Unmeasured Confounding

Human-Algorithm Interaction Biases in the Big Data Cycle: A Markov Chain Iterated Learning Framework

Modeling Contingent Decision Behavior: A Bayesian Nonparametric Preference-Learning Approach

Deliberating with AI: Improving Decision-Making for the Future through Participatory AI Design and Stakeholder Deliberation

A Voting-Based System for Ethical Decision Making

Learning to be Fair: A Consequentialist Approach to Equitable Decision-Making

StROL: Stabilized and Robust Online Learning from Humans

Behavior-Based Machine-Learning: A Hybrid Approach for Predicting Human Decision Making

The Principles and Limits of Algorithm-in-the-Loop Decision Making

Decision Learning : Data Analytic Learning with Strategic Decision Making