Abstract:Risk prediction models fitted using maximum likelihood estimation (MLE) are often overfitted resulting in predictions that are too extreme and a calibration slope (CS) less than 1. Penalized methods, such as Ridge and Lasso, have been suggested as a solution to this problem as they tend to shrink regression coefficients toward zero, resulting in predictions closer to the average. The amount of shrinkage is regulated by a tuning parameter, λ, commonly selected via cross‐validation ("standard tuning"). Though penalized methods have been found to improve calibration on average, they often over‐shrink and exhibit large variability in the selected λ and hence the CS. This is a problem, particularly for small sample sizes, but also when using sample sizes recommended to control overfitting. We consider whether these problems are partly due to selecting λ using cross‐validation with "training" datasets of reduced size compared to the original development sample, resulting in an over‐estimation of λ and, hence, excessive shrinkage. We propose a modified cross‐validation tuning method ("modified tuning"), which estimates λ from a pseudo‐development dataset obtained via bootstrapping from the original dataset, albeit of larger size, such that the resulting cross‐validation training datasets are of the same size as the original dataset. Modified tuning can be easily implemented in standard software and is closely related to bootstrap selection of the tuning parameter ("bootstrap tuning"). We evaluated modified and bootstrap tuning for Ridge and Lasso in simulated and real data using recommended sample sizes, and sizes slightly lower and higher. They substantially improved the selection of λ , resulting in improved CS compared to the standard tuning method. They also improved predictions compared to MLE.

A New Formula for Faster Computation of the K-Fold Cross-Validation and Good Regularisation Parameter Values in Ridge Regression

Cross-validatory Framework for Optimal Parameter Estimation of KPCA and KPLS Models

Fast Calculation of Gaussian Process Multiple-Fold Cross-Validation Residuals and their Covariances

Stability-Adjusted Cross-Validation for Sparse Linear Regression

Sketching the Krylov Subspace: Faster Computation of the Entire Ridge Regularization Path

Bayes beats Cross Validation: Efficient and Accurate Ridge Regression via Expectation Maximization

Fast cross-validation for multi-penalty ridge regression

Generalized Cross-Validation as a Method for Choosing a Good Ridge Parameter

Penalized Regression Methods With Modified Cross‐Validation and Bootstrap Tuning Produce Better Prediction Models

Solving Kernel Ridge Regression with Gradient-Based Optimization Methods

Have ASkotch: Fast Methods for Large-scale, Memory-constrained Kernel Ridge Regression

Gradient-based bilevel optimization for multi-penalty Ridge regression through matrix differential calculus

Kernel Truncated Randomized Ridge Regression: Optimal Rates and Low Noise Acceleration

A Theoretical Analysis of the Test Error of Finite-Rank Kernel Ridge Regression

A new ridge type estimator and its performance for the linear regression model: Simulation and application

Prevalidated ridge regression is a highly-efficient drop-in replacement for logistic regression for high-dimensional data

Fast Cross-Validation via Sequential Testing

A Comprehensive Analysis on the Learning Curve in Kernel Ridge Regression

An Agnostic View on the Cost of Overfitting in (Kernel) Ridge Regression

OKRidge: Scalable Optimal k-Sparse Ridge Regression

Optimal Subsampling for Large Sample Ridge Regression