Abstract:While machine learning algorithms hold promise for personalised medicine, their clinical adoption remains limited, partly due to biases that can compromise the reliability of predictions. In this paper, we focus on sample selection bias (SSB), a specific type of bias where the study population is less representative of the target population, leading to biased and potentially harmful decisions. Despite being well-known in the literature, SSB remains scarcely studied in machine learning for healthcare. Moreover, the existing machine learning techniques try to correct the bias mostly by balancing distributions between the study and the target populations, which may result in a loss of predictive performance. To address these problems, our study illustrates the potential risks associated with SSB by examining SSB's impact on the performance of machine learning algorithms. Most importantly, we propose a new research direction for addressing SSB, based on the target population identification rather than the bias correction. Specifically, we propose two independent networks(T-Net) and a multitasking network (MT-Net) for addressing SSB, where one network/task identifies the target subpopulation which is representative of the study population and the second makes predictions for the identified subpopulation. Our empirical results with synthetic and semi-synthetic datasets highlight that SSB can lead to a large drop in the performance of an algorithm for the target population as compared with the study population, as well as a substantial difference in the performance for the target subpopulations that are representative of the selected and the non-selected patients from the study population. Furthermore, our proposed techniques demonstrate robustness across various settings, including different dataset sizes, event rates, and selection rates, outperforming the existing bias correction techniques.

Addressing sample selection bias for machine learning methods

Correcting Classifiers for Sample Selection Bias in Two-Phase Case-Control Studies

Sample Selection Bias in Machine Learning for Healthcare

Systematic Bias in Sample Inference and its Effect on Machine Learning

Shedding light on underrepresentation and Sampling Bias in machine learning

Addressing bias in bagging and boosting regression models

Efficient estimation and correction of selection-induced bias with order statistics

A Bayesian algorithm for sample selection bias correction

De-Biasing Models of Biased Decisions: A Comparison of Methods Using Mortgage Application Data

Why Is My Classifier Discriminatory?

Propensity score adjustment using machine learning classification algorithms to control selection bias in online surveys

Addressing selection bias in cluster randomized experiments via weighting

Matching on Noise: Finite Sample Bias in the Synthetic Control Estimator

Adaptive Sampling Strategies to Construct Equitable Training Datasets

Equal Opportunity and Affirmative Action via Counterfactual Predictions

How to be fair? A study of label and selection bias

An Empirical Comparison of Bias Reduction Methods on Real-World Problems in High-Stakes Policy Settings

Learning Fair Policies for Multi-stage Selection Problems from Observational Data

Robust Design and Evaluation of Predictive Algorithms under Unobserved Confounding

Double machine learning and automated confounder selection: A cautionary tale

Bias in Machine Learning Software: Why? How? What to do?