Abstract:BACKGROUND: Social media have transformed the communications landscape. People increasingly obtain news and health information online and via social media. Social media platforms also serve as novel sources of rich observational data for health research (including infodemiology, infoveillance, and digital disease detection detection). While the number of studies using social data is growing rapidly, very few of these studies transparently outline their methods for collecting, filtering, and reporting those data. Keywords and search filters applied to social data form the lens through which researchers may observe what and how people communicate about a given topic. Without a properly focused lens, research conclusions may be biased or misleading. Standards of reporting data sources and quality are needed so that data scientists and consumers of social media research can evaluate and compare methods and findings across studies.OBJECTIVE: We aimed to develop and apply a framework of social media data collection and quality assessment and to propose a reporting standard, which researchers and reviewers may use to evaluate and compare the quality of social data across studies.METHODS: We propose a conceptual framework consisting of three major steps in collecting social media data: develop, apply, and validate search filters. This framework is based on two criteria: retrieval precision (how much of retrieved data is relevant) and retrieval recall (how much of the relevant data is retrieved). We then discuss two conditions that estimation of retrieval precision and recall rely on--accurate human coding and full data collection--and how to calculate these statistics in cases that deviate from the two ideal conditions. We then apply the framework on a real-world example using approximately 4 million tobacco-related tweets collected from the Twitter firehose.RESULTS: We developed and applied a search filter to retrieve e-cigarette-related tweets from the archive based on three keyword categories: devices, brands, and behavior. The search filter retrieved 82,205 e-cigarette-related tweets from the archive and was validated. Retrieval precision was calculated above 95% in all cases. Retrieval recall was 86% assuming ideal conditions (no human coding errors and full data collection), 75% when unretrieved messages could not be archived, 86% assuming no false negative errors by coders, and 93% allowing both false negative and false positive errors by human coders.CONCLUSIONS: This paper sets forth a conceptual framework for the filtering and quality evaluation of social data that addresses several common challenges and moves toward establishing a standard of reporting social data. Researchers should clearly delineate data sources, how data were accessed and collected, and the search filter building process and how retrieval precision and recall were calculated. The proposed framework can be adapted to other public social media platforms.

[How to Write High-Quality Epidemiological Research Paper Ⅵ. Strengthening the Reporting of Observational Studies in Epidemiology-Nutritional Epidemiology (Strobe-Nut)].

Strengthening the reporting of observational studies in epidemiology using mendelian randomisation (STROBE-MR): explanation and elaboration

The quality of reporting in clinical research: the CONSORT and STROBE initiatives

STROBE (STrengthening the Reporting of Observational studies in Epidemiology)

The reporting quality of meta-epidemiological studies needs substantial improvement: a research on research study

A Cross-Sectional Study of the Endorsement Proportion of Reporting Guidelines in 1039 Chinese Medical Journals

Behavioral intervention strategies for promoting healthy eating: perspectives from behavioral public health

Improving social justice in observational studies: protocol for the development of a global and Indigenous STROBE-equity reporting guideline

Quality of reporting web-based and non-web-based survey studies: What authors, reviewers and consumers should consider

Ten Points for High-Quality Statistical Reporting and Data Presentation

Enhancing the quality of reporting of orthodontic clinical research

Garbage in, Garbage Out: Data Collection, Quality Assessment and Reporting Standards for Social Media Data Use in Health Research, Infodemiology and Digital Disease Detection

Epidemiologic analyses with error-prone exposures: Review of current practice and recommendations

Spatial Lifecourse Epidemiology Reporting Standards (ISLE-ReSt) statement

Is reporting quality in medical publications associated with biostatisticians as co-authors? A registered report protocol

Reporting guidelines for human microbiome research: the STORMS checklist

Quality analysis of observational studies on pelvic organ prolapse in China

[Introduction of a Tool to Assess Risk of Bias in Non-randomized Studies-of Environmental Exposure (ROBINS-E)].

The reporting completeness of observational systematic reviews and meta-analysis in social science could be improved: a cross-sectional survey

The OHStat guidelines for reporting observational studies and clinical trials in oral health research: manuscript checklist