Abstract:BACKGROUND Recruitment of health research participants through social media is becoming more common. In the United States, 80% of adults use at least one social media platform. Social media platforms may allow researchers to reach potential participants efficiently. However, online research methods may be associated with unique threats to sample validity and data integrity. Limited research has described issues of data quality and authenticity associated with the recruitment of health research participants through social media, and sources of low-quality and fraudulent data in this context are poorly understood. OBJECTIVE The goal of the research was to describe and explain threats to sample validity and data integrity following recruitment of health research participants through social media and summarize recommended strategies to mitigate these threats. Our experience designing and implementing a research study using social media recruitment and online data collection serves as a case study. METHODS Using published strategies to preserve data integrity, we recruited participants to complete an online survey through the social media platforms Twitter and Facebook. Participants were to receive $15 upon survey completion. Prior to manually issuing remuneration, we reviewed completed surveys for indicators of fraudulent or low-quality data. Indicators attributable to respondent error were labeled suspicious, while those suggesting misrepresentation were labeled fraudulent. We planned to remove cases with 1 fraudulent indicator or at least 3 suspicious indicators. RESULTS Within 7 hours of survey activation, we received 271 completed surveys. We classified 94.5% (256/271) of cases as fraudulent and 5.5% (15/271) as suspicious. In total, 86.7% (235/271) provided inconsistent responses to verifiable items and 16.2% (44/271) exhibited evidence of bot automation. Of the fraudulent cases, 53.9% (138/256) provided a duplicate or unusual response to one or more open-ended items and 52.0% (133/256) exhibited evidence of inattention. CONCLUSIONS Research findings from several disciplines suggest studies in which research participants are recruited through social media are susceptible to data quality issues. Opportunistic individuals who use virtual private servers to fraudulently complete research surveys for profit may contribute to low-quality data. Strategies to preserve data integrity following research participant recruitment through social media are limited. Development and testing of novel strategies to prevent and detect fraud is a research priority.

Experiments in Social Media

Public attitudes towards social media field experiments

Facebook as a research tool for the social sciences: Opportunities, challenges, ethical considerations, and practical guidelines.

Big Data and the brave new world of social media research

A multilayered graph-based framework to explore behavioural phenomena in social media conversations

Evaluation of Biases in Self-reported Demographic and Psychometric Information: Traditional versus Facebook-based Surveys

Measuring Americans’ Comfort With Research Uses of Their Social Media Data

Automated Experimentation in Walden 3.0. : The Next step in Profiling, Predicting, Control and Surveillance

Living Labs - An Ethical Challenge for Researchers and Platform Providers

Psychological targeting as an effective approach to digital mass persuasion

How to Run Behavioural Experiments Online: Best Practice Suggestions for Cognitive Psychology and Neuroscience

Can social influence be exploited to compromise security: An online experimental evaluation

Social media advertising for clinical studies: Ethical and data protection implications of online targeting

Towards an Ethical Framework for Publishing Twitter Data in Social Research: Taking into Account Users’ Views, Online Context and Algorithmic Estimation

Big Questions for Social Media Big Data: Representativeness, Validity and Other Methodological Pitfalls

Why Don’t We Learn from Social Media? Studying Effects of and Mechanisms behind Social Media News Use on General Surveillance Political Knowledge

A 61-million-person experiment in social influence and political mobilization

Automating Dynamic Consent Decisions for the Processing of Social Media Data in Health Research

Gorilla in our Midst: An online behavioral experiment builder

Threats of Bots and Other Bad Actors to Data Quality Following Research Participant Recruitment Through Social Media: Cross-Sectional Questionnaire (Preprint)

Social Bots and Social Media Manipulation in 2020: The Year in Review