Book Review: Paul J. Lavrakes, Michael W. Traugott, Courtney Kennedy, Allyson L. Holbrook, Edith D. de Leeuw, and Brady West. Experimental Methods in Survey Research: Techniques that Combine Random Sampling with Random Assignment. 2019, Wiley, ISBN: 978-1-119-08374-0, 544 pages
K. J. Thompson
DOI: https://doi.org/10.2478/jos-2020-0046
2020-12-01
Journal of Official Statistics
Abstract:Qualitative and quantitative research are entrenched in survey methodology. Certainly, qualitative research tools that use small datasets such as focus groups and onsite visits of businesses provide invaluable information in assessing construct validity in survey instrument design, as does usability testing. However, these tools are limited, in the sense that they rely on observational data, are purposively gathered, and should not be used for any form of causal inference. At the opposite end of the spectrum, large amounts of data can be easily gathered via the internet using probability or nonprobability samples such as online convenience panels; the resulting statistics are time-dependent measures, generally considered to be externally valid. Again, however, these vehicles do not allow for causeand-effect analysis. Experimental Methods in Survey Research is an extensive collection of papers gathered together for one express purpose: to convince survey researchers to utilize experimental designs combining random assignment of subjects to treatment within random (probability) samples. Internal validity is attained via the random assignment of treatments, using blocking in the assignment process to control for “known” factors. Using a probability sample combined with successful recruitment methods should achieve external validity, assuming that the sampling frame covers the target population (i.e., has low coverage error) and that the response mechanism is ignorable (i.e., respondents are a random subsample). In assembling and organizing these chapters, the editors “embraced both the Campbell and Stanley validity framework” for research studies (statistical conclusion validity, construct validity, external validity, and internal validity) as well as the key components of the total survey error (TSE) framework, specifically coverage error, sampling error, nonresponse error, and measurement error. Thanks to the extensive usage of computer-assisted technologies in survey collection and the prevalence of internet access in the majority of target populations, the editors argue that there is little or no cost increase in embedding true experiments within ongoing programs, and therefore such tests should be carefully planned and executed on a wide scale in common practice. The book is divided into nine topical areas, comprising twenty-three chapters (the first chapter is an introduction). Each chapter covers one experimental research topic, providing a literature review and presenting at least one case study. Not perhaps
Computer Science