Abstract:Background: Increasingly, survey researchers rely on hybrid samples to improve coverage and increase the number of respondents by combining independent samples. For instance, it is possible to combine 2 probability samples with one relying on telephone and another on mail. More commonly, however, researchers are now supplementing probability samples with those from online panels that are less costly. Setting aside ad hoc approaches that are void of rigor, traditionally, the method of composite estimation has been used to blend results from different sample surveys. This means individual point estimates from different surveys are pooled together, 1 estimate at a time. Given that for a typical study many estimates must be produced, this piecemeal approach is computationally burdensome and subject to the inferential limitations of the individual surveys that are used in this process. Objective: In this paper, we will provide a comprehensive review of the traditional method of composite estimation. Subsequently, the method of composite weighting is introduced, which is significantly more efficient, both computationally and inferentially when pooling data from multiple surveys. With the growing interest in hybrid sampling alternatives, we hope to offer an accessible methodology for improving the efficiency of inferences from such sample surveys without sacrificing rigor. Methods: Specifically, we will illustrate why the many ad hoc procedures for blending survey data from multiple surveys are void of scientific integrity and subject to misleading inferences. Moreover, we will demonstrate how the traditional approach of composite estimation fails to offer a pragmatic and scalable solution in practice. By relying on theoretical and empirical justifications, in contrast, we will show how our proposed methodology of composite weighting is both scientifically sound and inferentially and computationally superior to the old method of composite estimation. Results: Using data from 3 large surveys that have relied on hybrid samples composed of probability-based and supplemental sample components from online panels, we illustrate that our proposed method of composite weighting is superior to the traditional method of composite estimation in 2 distinct ways. Computationally, it is vastly less demanding and hence more accessible for practitioners. Inferentially, it produces more efficient estimates with higher levels of external validity when pooling data from multiple surveys. Conclusions: The new realities of the digital age have brought about a number of resilient challenges for survey researchers, which in turn have exposed some of the inefficiencies associated with the traditional methods this community has relied upon for decades. The resilience of such challenges suggests that piecemeal approaches that may have limited applicability or restricted accessibility will prove to be inadequate and transient. It is from this perspective that our proposed method of composite weighting has aimed to introduce a durable and accessible solution for hybrid sample surveys.

[Weighted Estimation Methods for Multistage Sampling Survey Data].

Probability Weighted Clustered Coefficients Regression Models in Complex Survey Sampling

Model-based Inference for Two-Stage Cluster Samples Subject to Nonignorable Item Nonresponse

The Evaluation of Some Two-stage Sampling Methods

[Data Analysis from Surveys Using Complex Sampling Methods].

Bayesian Nonparametric Weighted Sampling Inference

Propensity score weighting for causal inference with multi-stage clustered data

On two-stage adaptive cluster sampling to assess pest density.

Self-Weighting Sample Design for Stratified Three-Stage or Over Sampling

Adjusted Logistic Propensity Weighting Methods for Population Inference using Nonprobability Volunteer-Based Epidemiologic Cohorts

Data Analysis Method in World Health Survey - China Survey

Improving the Efficiency of Inferences From Hybrid Samples for Effective Health Surveillance Surveys: Comprehensive Review of Quantitative Methods

Analysis of combined probability and nonprobability samples: A simulation evaluation and application to a teen smoking behavior survey

An Empirical Likelihood Approach to Data Analysis under Two-Stage Sampling Designs

Adjusted Confidence Bands for Complex Survey Data

Improving Survey Inference in Two-phase Designs Using Bayesian Machine Learning

On the implementation of stratified two-stage simple random sampling without replacement, with possible collapsed strata

Estimation and Variable Selection for Semiparametric Transformation Models under a More Efficient Cohort Sampling Design

Importance of sampling weights in multilevel modeling of international large-scale assessment data

Investigating an Alternative for Estimation from a Nonprobability Sample: Matching plus Calibration

Implementing machine learning methods with complex survey data: Lessons learned on the impacts of accounting sampling weights in gradient boosting