Abstract:Combining matching and regression for causal inference provides double-robustness in removing treatment effect estimation bias due to confounding variables. In most real-world applications, however, treatment and control populations are not large enough for matching to achieve perfect or near-perfect balance on all confounding variables and their nonlinear/interaction functions, leading to trade-offs. [this fact is independent of regression, so a bit disjointed from first sentence.] Furthermore, variance is as important of a contributor as bias towards total error in small samples, and must therefore be factored into the methodological decisions. In this paper, we develop a mathematical framework for quantifying the combined impact of matching and linear regression on bias and variance of treatment effect estimation. The framework includes expressions for bias and variance in a misspecified linear regression, theorems regarding impact of matching on bias and variance, and a constrained bias estimation approach for quantifying misspecification bias and combining it with variance to arrive at total error. Methodological decisions can thus be based on minimization of this total error, given the practitioner's assumption/belief about an intuitive parameter, which we call `omitted R-squared'. The proposed methodology excludes the outcome variable from analysis, thereby avoiding overfit creep and making it suitable for observational study designs. All core functions for bias and variance calculation, as well as diagnostic tools for bias-variance trade-off analysis, matching calibration, and power analysis are made available to researchers and practitioners through an open-source R library, MatchLinReg.

Estimating Simulation Metamodels Using Combined Correlation-Based Variance Reduction Techniques

Metamodel Estimation Using Integrated Correlation Methods.

Validation of Simulation Analysis Methods for the Schruben-Margolin Correlation-Induction Strategy

Simulation-Based Robust Design of Complex Product Considering Uncertainties of Metamodel, Design Variables, and Noise Parameters

Reducing Variance in a Crossdocking Simulation Model Using Common Random Numbers and Antithetic Variates

Validation of correlation-induction strategies for simulation experiments

Sensitivity Analysis of Expensive Black-Box Systems Using Metamodeling

Sensitivity analysis using the Metamodel of Optimal Prognosis

A method of combined metamodel and subset simulation for reliability analysis of rare events

Metamodel-based importance sampling for the simulation of rare events

Estimating outcome-specific effects in meta-analyses of multiple outcomes: A simulation study

Stochastic Kriging for Inadequate Simulation Models

Regression and Kriging metamodels with their experimental designs in simulation: A review

Multilevel Metamodels: A Novel Approach to Enhance Efficiency and Generalizability in Monte Carlo Simulation Studies

Combining matching and linear regression: Introducing a mathematical framework and software for simulations, diagnostics and calibration

Small sample adjustments for robust variance estimation with meta-regression.

Simulation Allocation for Determining the Best Design in the Presence of Correlated Sampling

Efficient global estimation of conditional-value-at-risk through stochastic kriging and extreme value theory

MODEL-BASED ANALYSIS TO IMPROVE THE PERFORMANCE OF ITERATIVE SIMULATIONS

Comparative study of metamodeling and sampling design for expensive and semi-expensive simulation models under uncertainty

Support vector regression based metamodeling for structural reliability analysis