Abstract:Regression models are used in geosciences to extrapolate data and identify significant predictors of a response variable. Criterion approaches based on the residual sum of squares (RSS), such as the Akaike Information Criterion, Bayesian Information Criterion (BIC), Deviance Information Criterion, or Mallows' Cp can be used to compare non-nested models to identify an optimal subset of covariates. Computational limitations arise when the number of observations or candidate covariates is large in comparing all possible combinations of the available covariates, and in characterizing the covariance of the residuals for each examined model when the residuals are autocorrelated, as is often the case in spatial and temporal regression analysis. This paper presents computationally efficient algorithms for identifying the optimal model as defined using any RSS-based model selection criterion. The proposed dual criterion optimal branch and bound (DCO B&B) algorithm is guaranteed to identify the optimal model, while a single criterion heuristic (SCH) B&B algorithm provides further computational savings and approximates the optimal solution. These algorithms are applicable both to multiple linear regression (MLR) and to response variables with correlated residuals. We also propose an approach for iterative model selection, where a single set of covariance parameters is used in each iteration rather than a different set of parameters being used for each examined model. Simulation experiments are performed to evaluate the performance of the algorithms for regression models, using MLR and geostatistical regression as prototypical regression tools and BIC as a prototypical model selection approach. Results show massive computational savings using the DCO B&B algorithm relative to performing an exhaustive search. The SCH B&B is shown to provide a good approximation of the optimal model in most cases, while the DCO B&B with iterative covariance parameter optimization yields the closest approximation to the DCO B&B algorithm while also providing additional computational savings.

Optimal Residuals and the Dahlquist Test Problem

On Designs for Recursive Least Squares Residuals to Detect Alternatives

A Population-Based Incremental Learning Method for Robust Optimal Solutions

Optimal Approximation of Unique Continuation

Pointwise Error Estimates of Numerical Solutions to Linear Quadratic Optimal Control Problems

Optimal Control Duality and the Douglas-Rachford Algorithm

Douglas-Rachford Algorithm for Control- and State-constrained Optimal Control Problems

A backward elimination discrete optimization algorithm for model selection in spatio-temporal regression models

A DPG method for linear quadratic optimal control problems

General Optimal Step-size for ADMM-type Algorithms: Domain Parametrization and Optimal Rates

Numerical Solution for Classical Optimal Control Problem Governing by Hyperbolic Partial Differential Equation via Galerkin Finite Element-Implicit method with Gradient Projection Method

Optimal Solutions of Well-Posed Linear Systems via Low-Precision Right-Preconditioned GMRES with Forward and Backward Stabilization

Backward error analysis and the qualitative behaviour of stochastic optimization algorithms: Application to stochastic coordinate descent

Solving Minimal Residual Methods in with Large Exponents p

Optimal Feedback for Stochastic Linear Quadratic Control and Backward Stochastic Riccati Equations in Infinite Dimensions

A Family of Iteration Functions for General Linear Systems

Optimal damping adapted to a set of initial conditions

Unconditionally optimal error estimate of a linearized variable-time-step BDF2 scheme for nonlinear parabolic equations

A new approach to proper orthogonal decomposition with difference quotients

A robust Kantorovich's theorem on inexact Newton method with relative residual error tolerance