Abstract:The aggregation of k-ary preferences is a historical and important problem, since it has many real-world applications, such as peer grading, presidential elections and restaurant ranking. Meanwhile, variants of Plackett-Luce model has been applied to aggregate k-ary preferences. However, there are two urgent issues still existing in the current variants. First, most of them ignore feature information. Namely, they consider k-ary preferences instead of instance-dependent k-ary preferences. Second, these variants barely consider the uncertainty in k-ary preferences provided by agnostic crowds. In this paper, we propose Deep plAckeTt-luce modEL wIth uNcertainty mEasurements (DATELINE), which can address both issues simultaneously. To address the first issue, we employ deep neural networks mapping each instance into its ranking score in Plackett-Luce model. Then, we present a weighted Plackett-Luce model to solve the second issue, where the weight is a dynamic uncertainty vector measuring the worker quality. More importantly, we provide theoretical guarantees for DATELINE to justify its robustness.

What problem does this paper attempt to address?

This paper attempts to solve two main problems existing in current variants of the Plackett - Luce model: 1. **Ignoring feature information**: Most existing variants of the Plackett - Luce model only consider the relative order between objects (i.e., object comparison) and ignore the instance feature information related to preferences. For example, when dealing with k - ary preferences of facial micro - expressions, the traditional Plackett - Luce model cannot consider high - dimensional features. 2. **Ignoring the uncertainty from unknown populations**: These models usually do not take into account the uncertainty existing in k - ary preferences provided by multiple different people. Since multiple people participate in the ranking process, noise may be introduced, thereby reducing the generalization ability of the traditional Plackett - Luce model. To solve the above problems, the author proposes a new model named Deep plAckeTt - luce mod ELwIth uNcertainty mEasurements (DATELINE). Specifically: - **For the first problem**, by using a deep neural network to map each instance to its ranking score in the Plackett - Luce model, object comparison and feature information are combined. \[ \lambda_{i, \theta}=\exp(f_\theta(x_i)), \quad i \in \{1, \dots, L\} \] - **For the second problem**, a weighted Plackett - Luce model is introduced, where the weights are dynamic uncertainty vectors used to measure the quality of workers. This uncertainty vector can be iteratively updated to adapt to k - ary noisy preferences. \[ P(\tilde{X}=\rho(i)|\lambda_\theta, \eta_w)=\sum_{t = i}^k \bar{\eta}_{w}^{(t - i + 1)}\delta(\lambda_{\rho(t)}, \theta) \] In addition, the author also provides theoretical guarantees (i.e., minimax rates) to prove the robustness of DATELINE. Through these improvements, DATELINE can be more accurate and robust when dealing with instance - dependent noisy preferences.

DATELINE: Deep Plackett-Luce Model with Uncertainty Measurements

A unified analysis of likelihood-based estimators in the Plackett--Luce model

Probabilistic linguistic q-rung orthopair fuzzy Archimedean aggregation operators for group decision-making

Joint Learning from Heterogeneous Rank Data

Robust risk aggregation with neural networks

Bounding Consideration Probabilities in Consider-Then-Choose Ranking Models

A New Probabilistic Model for Rank Aggregation.

Ranking Inferences Based on the Top Choice of Multiway Comparisons

(Un)certainty of (Un)fairness: Preference-Based Selection of Certainly Fair Decision-Makers

Choosing to Rank

Asymptotically Optimal Sequential Design for Rank Aggregation

Bayesian inferences on uncertain ranks and orderings: Application to ranking players and lineups

DebUnc: Mitigating Hallucinations in Large Language Model Agent Communication with Uncertainty Estimations

Risk measurement of aggregation approaches in multiple attribute decision making under uncertain information

A 2PLM-RANK multidimensional forced-choice model and its fast estimation algorithm

Analyzing Preference Data with Local Privacy: Optimal Utility and Enhanced Robustness

ON RANKING OF ALTERNATIVES IN UNCERTAIN GROUP DECISION MAKING MODEL

The Surprising Benefits of Base Rate Neglect in Robust Aggregation

Efficient Nearest Neighbor based Uncertainty Estimation for Natural Language Processing Tasks

MAGDM Linear-Programming Models with Distinct Uncertain Preference Structures.

A knapsack for collective decision-making