Abstract:Abstract Background For finite samples with binary outcomes penalized logistic regression such as ridge logistic regression has the potential of achieving smaller mean squared errors (MSE) of coefficients and predictions than maximum likelihood estimation. There is evidence, however, that ridge logistic regression can result in highly variable calibration slopes in small or sparse data situations. Methods In this paper, we elaborate this issue further by performing a comprehensive simulation study, investigating the performance of ridge logistic regression in terms of coefficients and predictions and comparing it to Firth’s correction that has been shown to perform well in low-dimensional settings. In addition to tuned ridge regression where the penalty strength is estimated from the data by minimizing some measure of the out-of-sample prediction error or information criterion, we also considered ridge regression with pre-specified degree of shrinkage. We included ‘oracle’ models in the simulation study in which the complexity parameter was chosen based on the true event probabilities (prediction oracle) or regression coefficients (explanation oracle) to demonstrate the capability of ridge regression if truth was known. Results Performance of ridge regression strongly depends on the choice of complexity parameter. As shown in our simulation and illustrated by a data example, values optimized in small or sparse datasets are negatively correlated with optimal values and suffer from substantial variability which translates into large MSE of coefficients and large variability of calibration slopes. In contrast, in our simulations pre-specifying the degree of shrinkage prior to fitting led to accurate coefficients and predictions even in non-ideal settings such as encountered in the context of rare outcomes or sparse predictors. Conclusions Applying tuned ridge regression in small or sparse datasets is problematic as it results in unstable coefficients and predictions. In contrast, determining the degree of shrinkage according to some meaningful prior assumptions about true effects has the potential to reduce bias and stabilize the estimates.

Identifying a class of Ridge-type estimators in binary logistic regression models

Modified ridge estimator in the Bell regression model

Positive-rule Stein-Type Almost Unbiased Ridge Estimator in Linear Regression Model

More on the Bias and Variance Comparisons of the Restricted Almost Unbiased Estimators

A new ridge type estimator and its performance for the linear regression model: Simulation and application

The Superiority of Bayes Linear Unbiased Minimum Variance Estimator with Respect to Ridge Estimator

The Research on Two Kinds of Restricted Biased Estimators Based on Mean Squared Error Matrix

Small-sample Properties for Generalized Ridge Estimator of Regression Coefficients in Linear Regression Model

On the Restricted Almost Unbiased Estimators in Linear Regression

Preliminary test almost unbiased ridge estimator in a linear regression model with multivariate Student-t errors

Liu Estimator in the Multinomial Logistic Regression Model

Liu-type Negative Binomial Regression: A Comparison of Recent Estimators and Applications

A New Ridge-Type Estimator in Stochastic Restricted Linear Regression

Linearized Ridge Regression Estimator under the Mean Squared Error Criterion in a Linear Regression Model

The R2 Ridge Trace in 2sls Regression Estimation

To tune or not to tune, a case study of ridge logistic regression in small or sparse datasets

A new stochastic mixed ridge estimator in linear regression model

Efficiency of a Stochastic Restricted Two-Parameter Estimator in Linear Regression

The Method to Determine the Ridge Parameter in Generalized Ridge-type Estimator

On the Weighted Mixed Almost Unbiased Ridge Estimator in Stochastic Restricted Linear Regression

Restricted Ridge Regression estimator as a parameter estimation in multiple linear regression model for multicollinearity case