Bias Reduction Using Mahalanobis-Metric Matching

Donald B. Rubin

DOI: https://doi.org/10.1002/j.2333-8504.1978.tb01164.x

2006-01-01

Abstract:SUMMARY Monte Carlo methods are used to study the ability of nearest available Mahalanobis metric matching to make the means of matching variables more similar in matched samples than in random samples.

What problem does this paper attempt to address?

MULTIVARIATE MATCHING METHODS THAT ARE EQUAL PERCENT BIAS REDUCING, I: SOME EXAMPLES

Donald B. Rubin

DOI: https://doi.org/10.2307/2529342

IF: 1.701

1976-01-01

Biometrics

Abstract:ABSTRACTMultivariate matching methods are commonly used in the behavioral and medical sciences in an attempt to control bias when randomization is not feasible. Some examples of multivariate matching methods are discussed in Althauser and Rubin (1970) and Cochran and Rubin (1973), but otherwise seem to have received little attention in the literature. Here, we present examples of multivariate matching methods that will yield the same percent reduction in bias for each matching variable for a variety of underlying distributions. Eleven distributional cases are considered, and for each one, matching methods are defined which are equal percent bias reducing. Methods discussed in Section 8, which are based on the values of the estimated best linear discriminant or which define distance by a sample based inner‐product, will probably be the most generally applicable in practice.
Matching to Remove Bias in Observational Studies

DB RUBIN

DOI: https://doi.org/10.2307/2529684

2006-01-01

Abstract:Several matching methods that match all of one sample from another larger sample on a continuous matching variable are compared with respect to their ability to remove the bias of the matching variable. One method is a simple mean-matching method and three are nearest available pair-matching methods. The methods' abilities to remove bias are also compared with the theoretical maximum given fixed distributions and fixed sample sizes. A summary of advice to an investigator is included.
MULTIVARIATE MATCHING METHODS THAT ARE EQUAL PERCENT BIAS REDUCING .2. MAXIMUMS ON BIAS REDUCTION FOR FIXED SAMPLE SIZES

Donald B. Rubin,DB RUBIN

DOI: https://doi.org/10.2307/2529343

IF: 1.701

1976-01-01

Biometrics

Abstract:Matched sampling is a method of data collection designed to reduce bias and variability due to specific matching variables. Although often used to control for bias in studies in which randomization is practically impossible, there is virtually no statistical literature devoted to investigating the ability of matched sampling to control bias in the common case of many matching variables. An obvious problem in studying the multivariate matching situation is the variety of sampling plans, underlying distributions, and intuitively reasonable matching methods. This article considers one class of multivariate matching methods which yield the same percent reduction in expected bias for each of the matching variables. The primary result is the derivation of the expression for the maximum attainable percent reduction in bias given fixed distributions and fixed sample sizes. An examination of trends in this maximum leads to a procedure for estimating minimum ratios of sample sizes needed to obtain well-matched samples.
MULTIVARIATE MATCHING METHODS THAT ARE EQUAL PERCENT BIAS REDUCING, II: MAXIMUMS ON BIAS REDUCTION FOR FIXED SAMPLE SIZES

Donald B. Rubin

DOI: https://doi.org/10.1002/j.2333-8504.1974.tb00866.x

1974-01-01

Research Bulletin

Abstract:ABSTRACTMatched sampling is a method of data collection designed to reduce bias and variability due to specific matching variables. Although often used to control for bias in studies in which randomization is practically impossible, there is virtually no statistical literature devoted to investigating the ability of matched sampling to control bias in the common case of many matching variables. An obvious problem in studying the multivariate matching situation is the variety of sampling plans, underlying distributions, and intuitively reasonable matching methods. This article considers one class of multivariate matching methods which yield the same percent reduction in expected bias for each of the matching variables. The primary result is the derivation of the expression for the maximum attainable percent reduction in bias given fixed distributions and fixed sample sizes. An examination of trends in this maximum leads to a procedure for estimating minimum ratios of sample sizes needed to obtain well‐matched samples.
The Bias Due to Incomplete Matching.

Paul R. Rosenbaum,Donald B. Rubin

DOI: https://doi.org/10.2307/2530647

IF: 1.701

1985-01-01

Biometrics

Abstract:Observational studies comparing groups of treated and control units are often used to estimate the effects caused by treatments. Matching is a method for sampling a large reservoir of potential controls to produce a control group of modest size that is ostensibly similar to the treated group. In practice, there is a trade-off between the desires to find matches for all treated units and to obtain matched treated-control pairs that are extremely similar to each other. We derive expressions for the bias in the average matched pair difference due to the failure to match all treated units--incomplete matching, and the failure to obtain exact matches--inexact matching. A practical example shows that the bias due to incomplete matching can be severe, and moreover, can be avoided entirely by using an appropriate multivariate nearest available matching algorithm, which, in the example, leaves only a small residual bias due to inexact matching.
Affinely Invariant Matching Methods with Ellipsoidal Distributions

Donald B. Rubin,Neal Thomas

DOI: https://doi.org/10.1214/aos/1176348671

1992-01-01

Abstract:Matched sampling is a common technique used for controlling bias in observational studies. We present a general theoretical framework for studying the performance of such matching methods. Specifically, results are obtained concerning the performance of affinely invariant matching methods with ellipsoidal distributions, which extend previous results on equal percent bias reducing methods. Additional extensions cover conditionally affinely invariant matching methods for covariates with conditionally ellipsoidal distributions. These results decompose the effects of matching into one subspace containing the best linear discriminant, and the subspace of variables uncorrelated with the discriminant. This characterization of the effects of matching provides a theoretical foundation for understanding the performance of specific methods such as matched sampling using estimated propensity scores. Calculations for such methods are given in subsequent articles.
The Use of Matched Sampling and Regression Adjustment to Remove Bias in Observational Studies

DB RUBIN

DOI: https://doi.org/10.2307/2529685

IF: 1.701

1973-01-01

Biometrics

Abstract:The ability of matched sampling and linear regression adjustment to reduce the bias of an estimate of the treatment effect in two sample observational studies is investigated for a simple matching method and five simple estimates. Monte Carlo results are given for moderately linear exponential response surfaces and analytic results are presented for quadratic response surfaces. The conclusions are (1) in general both matched sampling and regression adjustment can be expected to reduce bias, (2) in some cases when the variance of the matching variable differs in the two populations both matching and regression adjustment can increase bias, (3) when the variance of the matching variable is the same in the two populations and the distributions of the matching variable are symmetric the usual covariance adjusted estimate based on random samples is almost unbiased, and (4) the combination of regression adjustment in matched samples generally produces the least biased estimate.
Combining matching and linear regression: Introducing a mathematical framework and software for simulations, diagnostics and calibration

Alireza S. Mahani,Mansour T.A. Sharabiani

DOI: https://doi.org/10.48550/arXiv.1507.03159

2015-07-12

Abstract:Combining matching and regression for causal inference provides double-robustness in removing treatment effect estimation bias due to confounding variables. In most real-world applications, however, treatment and control populations are not large enough for matching to achieve perfect or near-perfect balance on all confounding variables and their nonlinear/interaction functions, leading to trade-offs. [this fact is independent of regression, so a bit disjointed from first sentence.] Furthermore, variance is as important of a contributor as bias towards total error in small samples, and must therefore be factored into the methodological decisions. In this paper, we develop a mathematical framework for quantifying the combined impact of matching and linear regression on bias and variance of treatment effect estimation. The framework includes expressions for bias and variance in a misspecified linear regression, theorems regarding impact of matching on bias and variance, and a constrained bias estimation approach for quantifying misspecification bias and combining it with variance to arrive at total error. Methodological decisions can thus be based on minimization of this total error, given the practitioner's assumption/belief about an intuitive parameter, which we call `omitted R-squared'. The proposed methodology excludes the outcome variable from analysis, thereby avoiding overfit creep and making it suitable for observational study designs. All core functions for bias and variance calculation, as well as diagnostic tools for bias-variance trade-off analysis, matching calibration, and power analysis are made available to researchers and practitioners through an open-source R library, MatchLinReg.

Methodology
An Empirical Comparison of Bias Reduction Methods on Real-World Problems in High-Stakes Policy Settings

Hemank Lamba,Kit T. Rodolfa,Rayid Ghani

DOI: https://doi.org/10.48550/arXiv.2105.06442

2021-05-14

Abstract:Applications of machine learning (ML) to high-stakes policy settings -- such as education, criminal justice, healthcare, and social service delivery -- have grown rapidly in recent years, sparking important conversations about how to ensure fair outcomes from these systems. The machine learning research community has responded to this challenge with a wide array of proposed fairness-enhancing strategies for ML models, but despite the large number of methods that have been developed, little empirical work exists evaluating these methods in real-world settings. Here, we seek to fill this research gap by investigating the performance of several methods that operate at different points in the ML pipeline across four real-world public policy and social good problems. Across these problems, we find a wide degree of variability and inconsistency in the ability of many of these methods to improve model fairness, but post-processing by choosing group-specific score thresholds consistently removes disparities, with important implications for both the ML research community and practitioners deploying machine learning to inform consequential policy decisions.

Machine Learning,Computers and Society
Matching Using Estimated Propensity Scores: Relating Theory to Practice

Donald B. Rubin,Neal Thomas

DOI: https://doi.org/10.2307/2533160

2006-01-01

Abstract:Matched sampling is a standard technique in the evaluation of treatments in observational studies. Matching on estimated propensity scores comprises an important class of procedures when there are numerous matching variables. Recent theoretical work (Rubin, D. B. and Thomas, N., 1992, The Annals of Statistics 20, 1079-1093) on affinely invariant matching methods with ellipsoidal distributions provides a general framework for evaluating the operating characteristics of such methods. Moreover, Rubin and Thomas (1992, Biometrika 79, 797-809) uses this framework to derive several analytic approximations under normality for the distribution of the first two moments of the matching variables in samples obtained by matching on estimated linear propensity scores. Here we provide a bridge between these theoretical approximations and actual practice. First, we complete and refine the nomal-based analytic approximations, thereby making it possible to apply these results to practice. Second, we perform Monte Carlo evaluations of the analytic results under normal and nonnormal ellipsoidal distributions, which confirm the accuracy of the analytic approximations, and demonstrate the predictable ways in which the approximations deviate from simulation results when normal assumptions are violated within the ellipsoidal family. Third, we apply the analytic approximations to real data with clearly nonellipsoidal distributions, and show that the theoretical expressions, although derived under artificial distributional conditions, produce useful guidance for practice. Our results delineate the wide range of settings in which matching on estimated linear propensity scores performs well, thereby providing useful information for the design of matching studies. When matching with a particular data set, our theoretical approximations provide benchmarks for expected performance under favorable conditions, thereby identifying matching variables requiring special treatment. After matching is complete and data analysis is at hand, our results provide the variances required to compute valid standard errors for common estimators.
Large Sample Properties of Matching for Balance

Yixin Wang,José R. Zubizarreta

DOI: https://doi.org/10.5705/ss.202020.0343

2021-09-12

Abstract:Matching methods are widely used for causal inference in observational studies. Among them, nearest neighbor matching is arguably the most popular. However, nearest neighbor matching does not generally yield an average treatment effect estimator that is $\sqrt{n}$-consistent (Abadie and Imbens, 2006). Are matching methods not $\sqrt{n}$-consistent in general? In this paper, we study a recent class of matching methods that use integer programming to directly target aggregate covariate balance as opposed to finding close neighbor matches. We show that under suitable conditions these methods can yield simple estimators that are $\sqrt{n}$-consistent and asymptotically optimal.

Statistics Theory,Methodology
Matching with multiple criteria and its application to health disparities research

Chang Chen,Zhiyu Qian,Bo Zhang

DOI: https://doi.org/10.48550/arXiv.2308.08217

2023-08-16

Abstract:Matching is a popular nonparametric covariate adjustment strategy in empirical health services research. Matching helps construct two groups comparable in many baseline covariates but different in some key aspects under investigation. In health disparities research, it is desirable to understand the contributions of various modifiable factors, like income and insurance type, to the observed disparity in access to health services between different groups. To single out the contributions from the factors of interest, we propose a statistical matching methodology that constructs nested matched comparison groups from, for instance, White men, that resemble the target group, for instance, black men, in some selected covariates while remaining identical to the white men population before matching in the remaining covariates. Using the proposed method, we investigated the disparity gaps between white men and black men in the US in prostate-specific antigen (PSA) screening based on the 2020 Behavioral Risk Factor Surveillance System (BFRSS) database. We found a widening PSA screening rate as the white matched comparison group increasingly resembles the black men group and quantified the contribution of modifiable factors like socioeconomic status. Finally, we provide code that replicates the case study and a tutorial that enables users to design customized matched comparison groups satisfying multiple criteria.

Applications
Scalable Large-Margin Mahalanobis Distance Metric Learning

Chunhua Shen,Junae Kim,Lei Wang

DOI: https://doi.org/10.1109/tnn.2010.2052630

2010-01-01

IEEE Transactions on Neural Networks

Abstract:For many machine learning algorithms such as k-nearest neighbor ( k-NN) classifiers and k-means clustering, often their success heavily depends on the metric used to calculate distances between different data points. An effective solution for defining such a metric is to learn it from a set of labeled training samples. In this work, we propose a fast and scalable algorithm to learn a Mahalanobis distance metric. The Mahalanobis metric can be viewed as the Euclidean distance metric on the input data that have been linearly transformed. By employing the principle of margin maximization to achieve better generalization performances, this algorithm formulates the metric learning as a convex optimization problem and a positive semidefinite (p.s.d.) matrix is the unknown variable. Based on an important theorem that a p.s.d. trace-one matrix can always be represented as a convex combination of multiple rank-one matrices, our algorithm accommodates any differentiable loss function and solves the resulting optimization problem using a specialized gradient descent procedure. During the course of optimization, the proposed algorithm maintains the positive semidefiniteness of the matrix variable that is essential for a Mahalanobis metric. Compared with conventional methods like standard interior-point algorithms or the special solver used in large margin nearest neighbor , our algorithm is much more efficient and has a better performance in scalability. Experiments on benchmark data sets suggest that, compared with state-of-the-art metric learning algorithms, our algorithm can achieve a comparable classification accuracy with reduced computational complexity.
A blended distance to define "people-like-me"

Anaïs Fopma,Mingyang Cai,Stef van Buuren,Gerko Vink

DOI: https://doi.org/10.48550/arXiv.2207.04650

2022-07-11

Methodology

Abstract:Curve matching is a prediction technique that relies on predictive mean matching, which matches donors that are most similar to a target based on the predictive distance. Even though this approach leads to high prediction accuracy, the predictive distance may make matches look unconvincing, as the profiles of the matched donors can substantially differ from the profile of the target. To counterbalance this, similarity between the curves of the donors and the target can be taken into account by combining the predictive distance with the Mahalanobis distance into a `blended distance' measure. The properties of this measure are evaluated in two simulation studies. Simulation study I evaluates the performance of the blended distance under different data-generating conditions. The results show that blending towards the Mahalanobis distance leads to worse performance in terms of bias, coverage, and predictive power. Simulation study II evaluates the blended metric in a setting where a single value is imputed. The results show that a property of blending is the bias-variance trade off. Giving more weight to the Mahalanobis distance leads to less variance in the imputations, but less accuracy as well. The main conclusion is that the high prediction accuracy achieved with the predictive distance necessitates the variability in the profiles of donors.
Matched Machine Learning: A Generalized Framework for Treatment Effect Inference With Learned Metrics

Marco Morucci,Cynthia Rudin,Alexander Volfovsky

DOI: https://doi.org/10.48550/arXiv.2304.01316

2023-04-04

Abstract:We introduce Matched Machine Learning, a framework that combines the flexibility of machine learning black boxes with the interpretability of matching, a longstanding tool in observational causal inference. Interpretability is paramount in many high-stakes application of causal inference. Current tools for nonparametric estimation of both average and individualized treatment effects are black-boxes that do not allow for human auditing of estimates. Our framework uses machine learning to learn an optimal metric for matching units and estimating outcomes, thus achieving the performance of machine learning black-boxes, while being interpretable. Our general framework encompasses several published works as special cases. We provide asymptotic inference theory for our proposed framework, enabling users to construct approximate confidence intervals around estimates of both individualized and average treatment effects. We show empirically that instances of Matched Machine Learning perform on par with black-box machine learning methods and better than existing matching methods for similar problems. Finally, in our application we show how Matched Machine Learning can be used to perform causal inference even when covariate data are highly complex: we study an image dataset, and produce high quality matches and estimates of treatment effects.

Methodology,Machine Learning
Demystifying Statistical Matching Algorithms for Big Data

Sanjeewani Weerasingha,Michael J. Higgins

DOI: https://doi.org/10.48550/arXiv.2309.05859

2023-09-12

Abstract:Statistical matching is an effective method for estimating causal effects in which treated units are paired with control units with ``similar'' values of confounding covariates prior to performing estimation. In this way, matching helps isolate the effect of treatment on response from effects due to the confounding covariates. While there are a large number of software packages to perform statistical matching, the algorithms and techniques used to solve statistical matching problems -- especially matching without replacement -- are not widely understood. In this paper, we describe in detail commonly-used algorithms and techniques for solving statistical matching problems. We focus in particular on the efficiency of these algorithms as the number of observations grow large. We advocate for the further development of statistical matching methods that impose and exploit ``sparsity'' -- by greatly restricting the available matches for a given treated unit -- as this may be critical to ensure scalability of matching methods as data sizes grow large.

Methodology
Combining Propensity Score Matching with Additional Adjustments for Prognostic Covariates

D Rubin,N Thomas

DOI: https://doi.org/10.2307/2669400

IF: 4.369

2000-01-01

Journal of the American Statistical Association

Abstract:Propensity score matching refers to a class of multivariate methods used in comparative studies to construct treated and matched control samples that have similar distributions on many covariates. This matching is the observational study analog of randomization in ideal experiments, but is far less complete as it can only balance the distribution of observed covariates, whereas randomization balances the distribution of all covariates, both observed and unobserved. An important feature of propensity score matching is that it can be easily combined with model-based regression adjustments or with matching on a subset of special prognostic covariates or combinations of prognostic covariates that have been identified as being especially predictive of the outcome variables. We extend earlier results by developing approximations for the distributions of covariates in matched samples created with linear propensity score methods for the practically important situation where matching uses both the estimated linear propensity scores and a set of special prognostic covariates. Such matching;on a subset of special prognostic covariates is an observational study analog of blocking in a randomized experiment. An example combining propensity score matching with Mahalanobis metric matching and regression adjustment is presented that demonstrates the flexibility of these methods for designing an observational study that effectively reduces both bias due to many observed covariates and bias and variability due to a more limited subset of covariates. Of particular importance, the general approach, which includes propensity score matching, was distinctly superior to methods that focus only on a subset of the prognostically most important covariates, even if those covariates account for most of the variation in the outcome variables. Also of importance, analyses based on matched samples were superior to those based on the full unmatched samples, even when regression adjustment was included.
A deterministic matching method for exact matchings to compare the outcome of different interventions

Felix Bestehorn,Maike Bestehorn,Christian Kirches

DOI: https://doi.org/10.48550/arXiv.2101.07029

2021-01-18

Abstract:Statistical matching methods are widely used in the social and health sciences to estimate causal effects using observational data. Often the objective is to find comparable groups with similar covariate distributions in a dataset, with the aim to reduce bias in a random experiment. We aim to develop a foundation for deterministic methods which provide results with low bias, while retaining interpretability. The proposed method matches on the covariates and calculates all possible maximal exact matchesfor a given dataset without adding numerical errors. Notable advantages of our method over existing matching algorithms are that all available information for exact matches is used, no additional bias is introduced, it can be combined with other matching methods for inexact matching to reduce pruning and that the result is calculated in a fast and deterministic way. For a given dataset the result is therefore provably unique for exact matches in the mathematical sense. We provide proofs, instructions for implementation as well as a numerical example calculated for comparison on a complete survey.

Applications
Alleviating Matching Bias in Marketing Recommendations.

Junpeng Fang,Qing Cui,Gongduo Zhang,Caizhi Tang,Lihong Gu,Longfei Li,Jinjie Gu,Jun Zhou,Fei Wu

DOI: https://doi.org/10.1145/3539618.3591854

2023-01-01

Abstract:In marketing recommendations, the campaign organizers will distribute coupons to users to encourage consumption. In general, a series of strategies are employed to interfere with the coupon distribution process, leading to a growing imbalance between user-coupon interactions, resulting in a bias in the estimation of conversion probabilities. We refer to the estimation bias as the matching bias. In this paper, we explore how to alleviate the matching bias from the causal-effect perspective. We regard the historical distributions of users and coupons over each other as confounders and characterize the matching bias as a confounding effect to reveal and eliminate the spurious correlations between user-coupon representations and conversion probabilities. Then we propose a new training paradigm named De-Matching Bias Recommendation (DMBR) to remove the confounding effects during model training via the backdoor adjustment. We instantiate DMBR on two representative models: DNN and MMOE, and conduct extensive offline and online experiments to demonstrate the effectiveness of our proposed paradigm.
Mistaken identities lead to missed opportunities: Testing for mean differences in partially matched data

Raymond Pomponio,Bailey K. Fosdick,Julia Wrobel,Ryan A. Peterson

DOI: https://doi.org/10.48550/arXiv.2312.14689

2023-12-22

Methodology

Abstract:It is increasingly common to collect pre-post data with pseudonyms or self-constructed identifiers. On survey responses from sensitive populations, identifiers may be made optional to encourage higher response rates. The ability to match responses between pre- and post-intervention phases for every participant may be impossible in such applications, leaving practitioners with a choice between the paired t-test on the matched samples and the two-sample t-test on all samples for evaluating mean differences. We demonstrate the inadequacies with both approaches, as the former test requires discarding unmatched data, while the latter test ignores correlation and assumes independence. In cases with a subset of matched samples, an opportunity to achieve limited inference about the correlation exists. We propose a novel technique for such `partially matched' data, which we refer to as the Quantile-based t-test for correlated samples, to assess mean differences using a conservative estimate of the correlation between responses based on the matched subset. Critically, our approach does not discard unmatched samples, nor does it assume independence. Our results demonstrate that the proposed method yields nominal Type I error probability while affording more power than existing approaches. Practitioners can readily adopt our approach with basic statistical programming software.

Bias Reduction Using Mahalanobis-Metric Matching

MULTIVARIATE MATCHING METHODS THAT ARE EQUAL PERCENT BIAS REDUCING, I: SOME EXAMPLES

Matching to Remove Bias in Observational Studies

MULTIVARIATE MATCHING METHODS THAT ARE EQUAL PERCENT BIAS REDUCING .2. MAXIMUMS ON BIAS REDUCTION FOR FIXED SAMPLE SIZES

MULTIVARIATE MATCHING METHODS THAT ARE EQUAL PERCENT BIAS REDUCING, II: MAXIMUMS ON BIAS REDUCTION FOR FIXED SAMPLE SIZES

The Bias Due to Incomplete Matching.

Affinely Invariant Matching Methods with Ellipsoidal Distributions

The Use of Matched Sampling and Regression Adjustment to Remove Bias in Observational Studies

Combining matching and linear regression: Introducing a mathematical framework and software for simulations, diagnostics and calibration

An Empirical Comparison of Bias Reduction Methods on Real-World Problems in High-Stakes Policy Settings

Matching Using Estimated Propensity Scores: Relating Theory to Practice

Large Sample Properties of Matching for Balance

Matching with multiple criteria and its application to health disparities research

Scalable Large-Margin Mahalanobis Distance Metric Learning

A blended distance to define "people-like-me"

Matched Machine Learning: A Generalized Framework for Treatment Effect Inference With Learned Metrics

Demystifying Statistical Matching Algorithms for Big Data

Combining Propensity Score Matching with Additional Adjustments for Prognostic Covariates

A deterministic matching method for exact matchings to compare the outcome of different interventions

Alleviating Matching Bias in Marketing Recommendations.

Mistaken identities lead to missed opportunities: Testing for mean differences in partially matched data