Identification, Impacts, and Opportunities of Three Common Measurement Considerations when using Digital Trace Data

Daniel Muise,Nilam Ram,Thomas Robinson,Byron Reeves

2023-09-30

Abstract:Cataloguing specific URLs, posts, and applications with digital traces is the new best practice for measuring media use and content consumption. Despite the apparent accuracy that comes with greater granularity, however, digital traces may introduce additional ambiguity and new errors into the measurement of media use. In this note, we identify three new measurement challenges when using Digital Trace Data that were recently uncovered using a new measurement framework - Screenomics - that records media use at the granularity of individual screenshots obtained every few seconds as people interact with mobile devices. We label the considerations as follows: (1) entangling - the common measurement error introduced by proxying exposure to content by exposure to format; (2) flattening - aggregating unique segments of media interaction without incorporating temporal information, most commonly intraindividually and (3) bundling - summation of the durations of segments of media interaction, indiscriminate with respect to variations across media segments.

Human-Computer Interaction,Computers and Society,Econometrics

What problem does this paper attempt to address?

The paper aims to address three common issues in the measurement of media use with Digital Trace Data (DTD) and proposes corresponding solutions. Specifically:

1. **Entangling**: The problem of conflating political content with its traditional format (such as news). Past research often measured political content bundled with news applications, but in reality, political content can come from various sources and media forms (such as social media posts, short videos, etc.). The study found that only a small portion of political content is delivered through news applications, so it is necessary to consider content and format separately.

2. **Flattening**: The problem of merging content fragments of different durations into a single metric. Current research typically sums up the browsing time of news of different lengths, ignoring the impact of duration on information processing. For example, political content fragments as short as a few seconds and those lasting several minutes are treated equally, whereas in reality, their impact on users may be completely different. The paper suggests considering the time factor in analysis to avoid simply merging fragments of different lengths.

3. **Bundling**: The problem of simply adding up the time information of multiple content fragments of different lengths. This approach ignores the actual differences between fragments, leading to aggregated data that cannot reflect the user's actual experience. For example, simply summing the time of all political content fragments would obscure the distinction between quick browsing and prolonged viewing. The study recommends considering the specific nature of fragments during aggregation to better understand their impact.

Overall, the paper emphasizes the need to consider time factors and content forms more meticulously when using digital trace data for media use measurement, to avoid measurement errors and improve the accuracy of research results.