Abstract:The receiver operating characteristic (ROC) curve is frequently used as a measure of accuracy of continuous markers in diagnostic tests. The area under the ROC curve (AUC) is arguably the most widely used summary index for the ROC curve. Although the small sample size scenario is common in medical tests, a comprehensive study of small sample size properties of various methods for the construction of the confidence/credible interval (CI) for the AUC has been by and large missing in the literature. In this paper, we describe and compare 29 non-parametric and parametric methods for the construction of the CI for the AUC when the number of available observations is small. The methods considered include not only those that have been widely adopted, but also those that have been less frequently mentioned or, to our knowledge, never applied to the AUC context. To compare different methods, we carried out a simulation study with data generated from binormal models with equal and unequal variances and from exponential models with various parameters and with equal and unequal small sample sizes. We found that the larger the true AUC value and the smaller the sample size, the larger the discrepancy among the results of different approaches. When the model is correctly specified, the parametric approaches tend to outperform the non-parametric ones. Moreover, in the non-parametric domain, we found that a method based on the Mann–Whitney statistic is in general superior to the others. We further elucidate potential issues and provide possible solutions to along with general guidance on the CI construction for the AUC when the sample size is small. Finally, we illustrate the utility of different methods through real life examples.

Semiparametric empirical likelihood confidence intervals for the difference of areas under two correlated ROC curves under density ratio model

Interval Estimation for the Difference in Paired Areas under the ROC Curves in the Absence of a Gold Standard Test

A Non-Parametric Method for the Comparison of Partial Areas under ROC Curves and Its Application to Large Health Care Data Sets.

A comparison of confidence/credible interval methods for the area under the ROC curve for continuous diagnostic tests with small sample size

Optimal two-phase sampling for comparing correlated areas under the ROC curves of two screening tests in the presence of verification bias

Nonparametric ROC Summary Statistics for Correlated Diagnostic Marker Data

Empirical Likelihood Based Inference for Conditional Pareto-type Tail Index

Empirical Likelihood Inference for Probability Density Functions under Association

Semiparametric ROC surface estimation for continuous diagnostic tests via polytomous logistic regression procedures

Nonparametric receiver operating characteristic curve analysis with an imperfect gold standard

Simultaneous inference for partial areas under receiver operating curves -- with a view towards efficiency

An optimal exact confidence interval for the difference of two independent binomial proportions

Confidence Bands for ROC Curves with Serially Dependent Data

Interpretation of the Area Under the ROC Curve for Risk Prediction Models

An efficient variance estimator of AUC and its applications to binary classification

Nonparametric Covariate Adjustment for Receiver Operating Characteristic Curves

A test for comparing conditional ROC curves with multidimensional covariates

Comparison of Diagnostic Markers with Repeated Measurements: a Non-Parametric ROC Curve Approach.

Accurate interval estimation for the risk difference in an incomplete correlated 2 × 2 table: Calf immunity analysis.

A Model for Adjusting for Nonignorable Verification Bias in Estimation of the ROC Curve and Its Area with Likelihood‐Based Approach

Two-Way Partial AUC and Its Properties