Abstract:The Bradley-Terry-Luce (BTL) model is one of the most widely used models for ranking a collection of items or agents based on pairwise comparisons among them. Given $n$ agents, the BTL model endows each agent $i$ with a latent skill score $\alpha_i > 0$ and posits that the probability that agent $i$ is preferred over agent $j$ is $\alpha_i/(\alpha_i + \alpha_j)$. In this work, our objective is to formulate a hypothesis test that determines whether a given pairwise comparison dataset, with $k$ comparisons per pair of agents, originates from an underlying BTL model. We formalize this testing problem in the minimax sense and define the critical threshold of the problem. We then establish upper bounds on the critical threshold for general induced observation graphs (satisfying mild assumptions) and develop lower bounds for complete induced graphs. Our bounds demonstrate that for complete induced graphs, the critical threshold scales as $\Theta((nk)^{-1/2})$ in a minimax sense. In particular, our test statistic for the upper bounds is based on a new approximation we derive for the separation distance between general pairwise comparison models and the class of BTL models. To further assess the performance of our statistical test, we prove upper bounds on the type I and type II probabilities of error. Much of our analysis is conducted within the context of a fixed observation graph structure, where the graph possesses certain ``nice'' properties, such as expansion and bounded principal ratio. Additionally, we derive several auxiliary results, such as bounds on principal ratios of graphs, $\ell^2$-bounds on BTL parameter estimation under model mismatch, stability of rankings under the BTL model, etc. We validate our theoretical results through experiments on synthetic and real-world datasets and propose a data-driven permutation testing approach to determine test thresholds.

Binary Hypothesis Testing for Softmax Models and Leverage Score Models

Attention Scheme Inspired Softmax Regression

The Closeness of In-Context Learning and Weight Shifting for Softmax Regression

MultiMax: Sparse and Multi-Modal Attention Learning

Exploring the Frontiers of Softmax: Provable Optimization, Applications in Diffusion Model, and Beyond

Maximum Score Change-Point Estimation in Binary Response Model

Two-Stage Maximum Score Estimator

Large Deviation Analysis of Score-Based Hypothesis Testing

In-Context Learning for Attention Scheme: from Single Softmax Regression to Multiple Softmax Regression via a Tensor Trick

Minimax Hypothesis Testing for the Bradley-Terry-Luce Model

A General Theory for Softmax Gating Multinomial Logistic Mixture of Experts

Sigsoftmax: Reanalysis of the Softmax Bottleneck

Finite-sample performance of the maximum likelihood estimator in logistic regression

Selective Boosting Algorithm for Maximizing the Soft Margin

Money on the Table: Statistical information ignored by Softmax can improve classifier accuracy

Large-Margin Regularized Softmax Cross-Entropy Loss

From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification

Superiority of Softmax: Unveiling the Performance Edge Over Linear Attention

Chi-square Loss for Softmax: an Echo of Neural Network Structure

Estimating the Probabilities of Rare Outputs in Language Models