Abstract:Calibration is a vital aspect of the performance of risk prediction models, but research in the context of ordinal outcomes is scarce. This study compared calibration measures for risk models predicting a discrete ordinal outcome, and investigated the impact of the proportional odds assumption on calibration and overfitting. We studied the multinomial, cumulative, adjacent category, continuation ratio, and stereotype logit/logistic models. To assess calibration, we investigated calibration intercepts and slopes, calibration plots, and the estimated calibration index. Using large sample simulations, we studied the performance of models for risk estimation under various conditions, assuming that the true model has either a multinomial logistic form or a cumulative logit proportional odds form. Small sample simulations were used to compare the tendency for overfitting between models. As a case study, we developed models to diagnose the degree of coronary artery disease (five categories) in symptomatic patients. When the true model was multinomial logistic, proportional odds models often yielded poor risk estimates, with calibration slopes deviating considerably from unity even on large model development datasets. The stereotype logistic model improved the calibration slope, but still provided biased risk estimates for individual patients. When the true model had a cumulative logit proportional odds form, multinomial logistic regression provided biased risk estimates, although these biases were modest. Non-proportional odds models require more parameters to be estimated from the data, and hence suffered more from overfitting. Despite larger sample size requirements, we generally recommend multinomial logistic regression for risk prediction modeling of discrete ordinal outcomes.

Stochastic orders and shape properties for a new distorted proportional odds model

Hessian and increasing-Hessian orderings of scale-shape mixtures of multivariate skew-normal distributions and applications

Non Proportional Odds Models are Widely Dispensable -- Sparser Modeling based on Parametric and Additive Location-Shift Approaches

The generalized odd log-logistic family of distributions: properties, regression models and applications

On the Mixture of Proportional Odds Models

A new class of estimators for the shape parameter of a Pareto model

A new regression model for rates and proportions data with applications

Stochastic orders and distortion risk contribution ratio measures

Reliability study of proportional odds family of discrete distributions

The Fragility Of Standard Inferential Approaches In Principal Stratification Models Relative To Direct Likelihood Approaches

PROPORTIONAL HAZARDS MODELS WITH CONTINUOUS MARKS

Stochastic comparisons of second-order statistics from dependent and heterogeneous modified proportional (reversed) hazard rates scale models

The parametric and additive partial linear regressions based on the generalized odd log-logistic log-normal distribution

Optimal allocation of policy limits and deductibles under distortion risk measures

Orderings of the finite mixture with modified proportional hazard rate model

The bounded rationality of probability distortion.

On the Skewness of Order Statistics in Multiple-Outlier PHR Models

A Transformation Perspective on Marginal and Conditional Models

A novel distribution with upside down bathtub shape hazard rate: properties, estimation and applications

Statistical inference for counting processes under shape heterogeneity

Risk prediction models for discrete ordinal outcomes: calibration and the impact of the proportional odds assumption