Abstract:Regression models are used in geosciences to extrapolate data and identify significant predictors of a response variable. Criterion approaches based on the residual sum of squares (RSS), such as the Akaike Information Criterion, Bayesian Information Criterion (BIC), Deviance Information Criterion, or Mallows' Cp can be used to compare non-nested models to identify an optimal subset of covariates. Computational limitations arise when the number of observations or candidate covariates is large in comparing all possible combinations of the available covariates, and in characterizing the covariance of the residuals for each examined model when the residuals are autocorrelated, as is often the case in spatial and temporal regression analysis. This paper presents computationally efficient algorithms for identifying the optimal model as defined using any RSS-based model selection criterion. The proposed dual criterion optimal branch and bound (DCO B&B) algorithm is guaranteed to identify the optimal model, while a single criterion heuristic (SCH) B&B algorithm provides further computational savings and approximates the optimal solution. These algorithms are applicable both to multiple linear regression (MLR) and to response variables with correlated residuals. We also propose an approach for iterative model selection, where a single set of covariance parameters is used in each iteration rather than a different set of parameters being used for each examined model. Simulation experiments are performed to evaluate the performance of the algorithms for regression models, using MLR and geostatistical regression as prototypical regression tools and BIC as a prototypical model selection approach. Results show massive computational savings using the DCO B&B algorithm relative to performing an exhaustive search. The SCH B&B is shown to provide a good approximation of the optimal model in most cases, while the DCO B&B with iterative covariance parameter optimization yields the closest approximation to the DCO B&B algorithm while also providing additional computational savings.

Optimizing control variable selection with algorithms: Parsimony and precision in regression analysis

Bayesian variable selection in linear regression models with instrumental variables

New Algorithms and Software for Significance Controlled Variable Selection

A tourist in the economics of tourism: Reflections on nonparametric estimation of stochastic frontier models

Control Variables in Leadership Research: A Qualitative and Quantitative Review

A Data-driven Approach for Selecting Controlled Variables

Variable Selection with Stepwise and Best Subset Approaches.

Mastering the Use of Control Variables: the Hierarchical Iterative Control (HIC) Approach

A backward elimination discrete optimization algorithm for model selection in spatio-temporal regression models

Economic variable selection

Purposeful selection of variables in logistic regression

Variable Selection via Adaptive False Negative Control in Linear Regression

Residuals and Regression Diagnostics: Focusing on Logistic Regression

Controlled Variable Selection from Summary Statistics Only? A Solution via GhostKnockoffs and Penalized Regression

Comparing Stochastic Optimization Methods for Variable Selection in Binary Outcome Prediction, With Application to Health Policy

Variable selection with error control: Another look at Stability Selection

LOOKAHEAD AND PILOTING STRATEGIES FOR VARIABLE SELECTION

Variable Selection in Macroeconomic Forecasting with Many Predictors

Variable Selection for Linear Mixed Effects Model Via Penalization Approaches

An efficient optimization approach for best subset selection in linear regression, with application to model selection and fitting in autoregressive time-series

Combining Observational and Experimental Data to Improve Efficiency Using Imperfect Instruments