Abstract:Prior to clinical applications, it is critical that risk prediction models are evaluated in independent studies that did not contribute to model development. While prospective cohort studies provide a natural setting for model validation, they often ascertain information on some risk factors (e.g., an expensive biomarker) in a nested sub-study of the original cohort, typically selected based on case-control status, and possibly some additional covariates. In this article, we propose an efficient approach for evaluating discriminatory ability of models using data from all individuals in a cohort study irrespective of whether they were sampled in the nested sub-study for measuring the complete set of risk factors. For evaluation of the Area Under the Curve (AUC) statistics, we estimate probabilities of risk-scores for cases being larger than those in controls conditional on partial risk-scores, the component of the risk-score that could be defined based on partial covariate information. The use of partial risk-scores, as opposed to actual multivariate risk-factor profiles, allows estimation of the underlying conditional expectations using subjects with complete covariate information in a non-parametric fashion even when numerous covariates are involved. We propose an influence function based approach for estimation of the variance of the resulting AUC statistics. We evaluate finite sample performance of the proposed method and compare it to an inverse probability weighted (IPW) estimator through extensive simulation studies. Finally, we illustrate an application of the proposed method for evaluating performance of a lung cancer risk prediction model using data from the Prostate, Lung, Colorectal and Ovarian Cancer (PLCO) trial.

Discrimination for Two Way Models with Insurance Application

PENALIZED VARIABLE SELECTION PROCEDURE FOR COX MODELS WITH SEMIPARAMETRIC RELATIVE RISK

Model Uncertainty and Selection of Risk Models for Left-Truncated and Right-Censored Loss Data

Scale-mixture Birnbaum-Saunders quantile regression models applied to personal accident insurance data

A comparative analysis of several multivariate zero-inflated and zero-modified models with applications in insurance

Econometrics of Insurance with Multidimensional Types

Identification of Insurance Models with Multidimensional Screening

Bayesian variable selection in linear regression models with instrumental variables

Evaluation of Model Selection Strategies for Cross-Level Two-Way Differential Item Functioning Analysis

A Two-Stage Model for High-Risk Prediction in Insurance Ratemaking: Asymptotics and Inference

High Dimensional Binary Choice Model with Unknown Heteroskedasticity or Instrumental Variables

Skewed link regression models for imbalanced binary response with applications to life insurance

Application of a General Family of Bivariate Distributions in Modelling Dependent Competing Risks Data with Associated Model Selection

A Posteriori Risk Classification and Ratemaking with Random Effects in the Mixture-of-Experts Model

Bivariate Gamma Mixture of Experts Models for Joint Insurance Claims Modeling

Penalized Variable Selection with Broken Adaptive Ridge Regression for Semi-competing Risks Data

Bivariate phase-type distributions for experience rating in disability insurance

Mitigating Discrimination in Insurance with Wasserstein Barycenters

Bayesian Variable Selection in Double Generalized Linear Tweedie Spatial Process Models

Evaluating discriminatory accuracy of models using partial risk-scores in two-phase studies

Risk Preference Types, Limited Consideration, and Welfare