Spillover Effects in Online Field Experiments

Yuan,Tracy Xiao Liu

DOI: https://doi.org/10.1145/3572885.3572893

2022-01-01

Abstract:Here we provide an overview of an important issue in online field experiments: spillover effects. We include a reading list for researchers in both academia and industry who are interested in this topic.

What problem does this paper attempt to address?

Spillover Effects in Online Field Experiments: An Annotated Reading List

Yuan Yuan,Tracy Xiao Liu

DOI: https://doi.org/10.1145/3572885.3572893

2022-01-01

ACM SIGecom Exchanges

Abstract:Here we provide an overview of an important issue in online field experiments: spillover effects. We include a reading list for researchers in both academia and industry who are interested in this topic.
Detecting Spillover Effects: Design and Analysis of Multilevel Experiments

Betsy Sinclair,Margaret McConnell,Donald P. Green

DOI: https://doi.org/10.1111/j.1540-5907.2012.00592.x

IF: 4.503

2012-05-03

American Journal of Political Science

Abstract:Interpersonal communication presents a methodological challenge and a research opportunity for researchers involved in field experiments. The challenge is that communication among subjects blurs the line between treatment and control conditions. When treatment effects are transmitted from subject to subject, the stable unit treatment value assumption (SUTVA) is violated, and comparison of treatment and control outcomes may provide a biased assessment of the treatment’s causal influence. Social scientists are increasingly interested in the substantive phenomena that lead to SUTVA violations, such as communication in advance of an election. Experimental designs that gauge SUTVA violations provide useful insights into the extent and influence of interpersonal communication. This article illustrates the value of one such design, a multilevel experiment in which treatments are randomly assigned to individuals and varying proportions of their neighbors. After describing the theoretical and statistical underpinnings of this design, we apply it to a large‐scale voter‐mobilization experiment conducted in Chicago during a special election in 2009 using social‐pressure mailings that highlight individual electoral participation. We find some evidence of within‐household spillovers but no evidence of spillovers across households. We conclude by discussing how multilevel designs might be employed in other substantive domains, such as the study of deterrence and policy diffusion.
Is Attention Contagious? Estimating The Spillover Effect Of Investor Attention In Digital Networks

Wuyue (Phoebe) Shangguan,Xi Chen,Alvin Chung Man Leung

2020-01-01

Abstract:This study constructs networks based on visitors' online co-searches of firms to explore the economic value of visible network linkages on digital platforms. To achieve this, we investigate whether exogenous attention shocks of some firms can diffuse through network linkages and spill over to proximate firms. We design a quasi-experiment by leveraging the attention shocks based on multiple external sources and employ an identification strategy based on difference-in-differences models with propensity score-based matching. We find strong evidence on the existence of attention spillover. Both the abnormal return and risk of neighboring firms will significantly increase after "catching" the contagion. Besides, the spillover effect persists only in a short time window and decays over time and the distance from the sources of attention shocks. Finally, we find heterogeneous spillover effects across firms: firms with small size or negative public sentiment are more susceptible to the contagion.
Online or offline? Spillover effect of customer-to-customer interaction in a multichannel background

Yucheng Liu,Xiaorong Fu,Xiangming Ren

DOI: https://doi.org/10.1108/intr-11-2021-0855

IF: 5.9

2023-01-05

Internet Research

Abstract:Purpose Enterprises' multichannel operations provide various avenues for customer interaction; however, existing literature investigating customer-to-customer interaction (CCI) mainly focuses on a single channel. The purpose of this paper is to investigate the spillover effect of CCI and potential underlying mediating mechanisms in different information channels. Design/methodology/approach Three between-subjects experiments with 946 participants were employed to empirically validate the proposed hypotheses in the context of an experiential product and a material product. Findings Results suggest the clear spillover effect of CCI, indicating that positive CCI improves focal customers' satisfaction and purchase intention, whereas negative CCI reduces focal customers' satisfaction and purchase intention. Moreover, CCI's spillover effect varies based on the CCI channel. Offline CCI has a stronger positive spillover effect than online CCI. Contrarily, online CCI has a stronger negative spillover effect than offline CCI. Customer experience and trust are demonstrated to have mediating roles in this process. Originality/value This study is the first to comprehensively understand and compare the CCI spillover effect of the two information channels. The findings add to the existing knowledge of information processing in the psychological mechanisms influencing the belief in addition to providing insights for companies engaged in multichannel operations management across different channels.

computer science, information systems,telecommunications,business
Spillover effects in environmental behaviors, across time and context: a review and research agenda

Andreas Nilsson,Magnus Bergquist,Wesley P. Schultz

DOI: https://doi.org/10.1080/13504622.2016.1250148

IF: 3.725

2016-11-02

Environmental Education Research

Abstract:When implementing environmental education and interventions to promote one pro-environmental behavior, it is seldom asked if and how non-target pro-environmental behaviors are affected. The spillover effect proposes that engaging in one behavior affects the probability of engagement or disengaging in a second behavior. Therefore, the positive spillover effect predicts that interventions targeting one specific behavioral have the capacity to promote non-targeted and/or future pro-environmental behaviors. However, the negative spillover effect predicts that engaging in a first pro-environmental behavior will prevent or decrease a second pro-environmental behavior. Since the theoretical and empirical basis for positive and negative spillover effects are not sufficiently understood, the present paper (1) suggests a distinction between behavioral, temporal, and contextual spillovers (2) reviews the existing spillover research literature across a variety of environmental domains, (3) presents potential moderators governing the direction of spillover effects, and finally (4) discuss techniques to promote pro-environmental spillovers.

environmental studies,education & educational research
Beyond the Targeted Customer: Spillover Effect Through Social Influence

Zhen-Zhen Wang,Zhihua Zhu,Xiaoke Xu,Naipeng Chao

DOI: https://doi.org/10.2501/jar-2023-016

IF: 3.031

2023-01-01

Journal of Advertising Research

Abstract:Following an innovative large-scale field experiment, this study measures the extent of the spillover effect through social influence in a social advertising campaign. Results show that, even when the campaign does not have a significant direct effect on targeted customers, it still creates a significant spillover effect on targets’ peers because of social influence. The spillover effect is achieved through weak ties (acquaintances) rather than strong ones (close relationships). For social advertising, it may be more effective to target those with broad weak connections rather than those with strong ties.
Effects of online-to-offline spillovers on pricing and quality strategies of competing firms

Shuang Dong,Zhongfeng Qin,Yingchen Yan

DOI: https://doi.org/10.1016/j.ijpe.2021.108376

IF: 11.251

2022-02-01

International Journal of Production Economics

Abstract:With the rapid development of online retail, many manufacturers selling through offline channels have also ventured into online selling. However, much literature indicates that these omnichannel sales, relative to single-channel sales, will lead to cross-channel spillovers, that is, the stimulation or cannibalization effect of online sales on offline sales. This paper studies the spillover effect on the price and quality decisions of competing manufacturers and how these manufacturers initially respond to the cross-channel spillovers by adjusting product strategies. In the model, an incumbent already sells product in online market and an entrant entering the market with its higher-quality product. Meanwhile, firms may have been selling through their respective offline channels. Results indicate that although the positive spillover of online sales stimulates offline sales, it strengthens online competition by reducing the equilibrium prices. Moreover, a spillover can enhance or reduce the entrant's quality level. Taking positive spillover for example, a firm prefers to enhance its product quality when its spillover is significantly smaller than its competitor's or when its spillover is slightly larger but its product quality improvement is limited. Moreover, we find that the entrant's (incumbent's) quality improvement can alleviate (strengthen) the effect of a spillover, which provides firms useful insights into managing spillovers.

engineering, manufacturing, industrial,operations research & management science
Identification and estimation of spillover effects in randomized experiments

Gonzalo Vazquez-Bare

DOI: https://doi.org/10.1016/j.jeconom.2021.10.014

IF: 3.363

2022-01-01

Journal of Econometrics

Abstract:I study identification, estimation and inference for spillover effects in experiments where units’ outcomes may depend on the treatment assignments of other units within a group. I show that the commonly-used reduced-form linear-in-means regression identifies a weighted sum of spillover effects with some negative weights, and that the difference in means between treated and controls identifies a combination of direct and spillover effects entering with different signs. I propose nonparametric estimators for average direct and spillover effects that overcome these issues and are consistent and asymptotically normal under a precise relationship between the number of parameters of interest, the total sample size and the treatment assignment mechanism. These findings are illustrated using data from a conditional cash transfer program and with simulations. The empirical results reveal the potential pitfalls of failing to flexibly account for spillover effects in policy evaluation: the estimated difference in means and the reduced-form linear-in-means coefficients are all close to zero and statistically insignificant, whereas the nonparametric estimators I propose reveal large, nonlinear and significant spillover effects.

economics,social sciences, mathematical methods,mathematics, interdisciplinary applications
Programme evaluation and spillover effects

M. Angelucci,V. Di Maro

DOI: https://doi.org/10.1080/19439342.2015.1033441

2015-04-20

Journal of Development Effectiveness

Abstract:This paper is a practical guide for researchers and practitioners who want to understand spillover effects in programme evaluation. It defines spillover effects and discusses why it is important to measure them. It explains how to design a field experiment to measure the average effects of the treatment on subjects, both eligible and ineligible, for the programme in the presence of spillover effects. In addition, it discusses the use of nonexperimental methods for estimating spillover effects when the experimental design is not a viable option. Evaluations that account for spillover effects should be designed such that they explain both the cause of these effects and whom they affect. Such an evaluation design is necessary to avoid inappropriate policy recommendations and neglecting important mechanisms through which the programme operates.

development studies
Spillover modes in multiplex games: double-edged effects on cooperation, and their coevolution

Tommy Khoo,Feng Fu,Scott Pauls

DOI: https://doi.org/10.48550/arXiv.1710.03356

2017-10-10

Abstract:In recent years, there has been growing interest in studying games on multiplex networks that account for interactions across linked social contexts. However, little is known about how potential cross-context interference, or spillover, of individual behavioural strategy impact overall cooperation. We consider three plausible spillover modes, quantifying and comparing their effects on the evolution of cooperation. In our model, social interactions take place on two network layers: one represents repeated interactions with close neighbours in a lattice, the other represents one-shot interactions with random individuals across the same population. Spillover can occur during the social learning process with accidental cross-layer strategy transfer, or during social interactions with errors in implementation due to contextual interference. Our analytical results, using extended pair approximation, are in good agreement with extensive simulations. We find double-edged effects of spillover on cooperation: increasing the intensity of spillover can promote cooperation provided cooperation is favoured in one layer, but too much spillover is detrimental. We also discover a bistability phenomenon of cooperation: spillover hinders or promotes cooperation depending on initial frequencies of cooperation in each layer. Furthermore, comparing strategy combinations that emerge in each spillover mode provides a good indication of their co-evolutionary dynamics with cooperation. Our results make testable predictions that inspire future research, and sheds light on human cooperation across social domains and their interference with one another.

Physics and Society
Online Field Experiments: a Selective Survey of Methods

Yan Chen,Joseph Konstan

DOI: https://doi.org/10.1007/s40881-015-0005-3

2015-01-01

Journal of the Economic Science Association

Abstract:The Internet presents today’s researchers with unprecedented opportunities to conduct field experiments. Using examples from Economics and Computer Science, we present an analysis of the design choices, with particular attention to the underlying technologies, in conducting online field experiments and report on lessons learned.
Spillover Effect of Content Marketing in E-commerce Platform under the Fan Economy Era.

Ruibin Geng,Shichao Wang,Xi Chen

2018-01-01

Abstract:As the proliferation of social media and live streaming, online celebrity endorsement is a common practice of content marketing in e-commerce platform. Despite the prevalent use of social media and online community, empirical research investigating the economic values of user-generated-content (UGC) and marketer-generated-content (MGC) still lags. This study seeks to contribute theoretically and practically to an understanding of how online celebrity endorsement and fans interaction behaviors affect e-commerce sales. We adopt cross-sectional regression to assess the economic value of online celebrity endorsement, and we employ panel vector autoregressive model to explain the dynamic relationship between marketers’ and consumers’ content marketing behaviors and e-commerce product sales. Empirical results highlight that the interaction within fans community has spillover effect on content marketing under “Fan Economy” era.
Beyond the Coverage of Information Spreading: Analytical and Empirical Evidence of Re-exposure in Large-scale Online Social Networks

Xin Lu,Shuo Qin,Petter Holme,Fanhui Meng,Yanqing Hu,Fredrik Liljeros,Gad Allon

DOI: https://doi.org/10.48550/arXiv.1907.12389

2019-07-26

Abstract:Peer influence and social contagion are key denominators in the adoption and participation of information spreading, such as news propagation, word-of-mouth or viral marketing. In this study, we argue that it is biased to only focus on the scale and coverage of information spreading, and propose that the level of influence reinforcement, quantified by the re-exposure rate, i.e., the rate of individuals who are repeatedly exposed to the same information, should be considered together to measure the effectiveness of spreading. We show that local network structural characteristics significantly affects the probability of being exposed or re-exposed to the same information. After analyzing trending news on the super large-scale online network of Sina Weibo (China's Twitter) with 430 million connected users, we find a class of users with extremely low exposure rate, even they are following tens of thousands of others; and the re-exposure rate is substantially higher for news with more transmission waves and stronger secondary forwarding. While exposure and re-exposure rate typically grow together with the scale of spreading, we find exceptional cases where it is possible to achieve a high exposure rate while maintaining low re-exposure rate, or vice versa.

Physics and Society,Applications
Targeted Promotions and Cross-Category Spillover Effects

Nathan M. Fong,Yuchi Zhang,Xueming Luo,Xiaoyi Wang

DOI: https://doi.org/10.2139/ssrn.2847635

2016-01-01

Abstract:Targeted promotions based on individual purchase histories are known to increase promotional response while reducing search effort, yet the opportunity costs of such targeted promotions remain poorly understood. We use field experiments on a mobile e-book reading app and an online ticket exchange to test how targeted promotions can decrease cross-category search and purchasing. Findings from three randomized field experiments consistently showed that targeted promotions reduce cross-category purchase activity. These negative spillovers can be large; while direct response was higher for targeted promotions, total sales for one of the experiments was significantly higher for untargeted promotions. We find evidence that the negative category spillover effects are due to a decrease in search activity. Precisely targeted offers lead consumers to take a less active role in information search, limiting cross-category exploration. Firms using highly targeted promotions should monitor for reductions in search, as an indicator of missed cross-selling opportunities and lower sales diversity at the customer level.
How Events Determine Spreading Patterns: Information Transmission Via Internal and External Influences on Social Networks

Chuang Liu,Xiu-Xiu Zhan,Zi-Ke Zhang,Gui-Quan Sun,Pak Ming Hui

DOI: https://doi.org/10.1088/1367-2630/17/11/113045

2015-01-01

New Journal of Physics

Abstract:Recently, information transmission models motivated by the classical epidemic propagation, have been applied to a wide-range of social systems, generally assume that information mainly transmits among individuals via peer-to-peer interactions on social networks. In this paper, we consider one more approach for users to get information: the out-of-social-network influence. Empirical analyzes of eight typical events' diffusion on a very large micro-blogging system, Sina Weibo, show that the external influence has significant impact on information spreading along with social activities. In addition, we propose a theoretical model to interpret the spreading process via both internal and external channels, considering three essential properties: (i) memory effect; (ii) role of spreaders; and (iii) non-redundancy of contacts. Experimental and mathematical results indicate that the information indeed spreads much quicker and broader with mutual effects of the internal and external influences. More importantly, the present model reveals that the event characteristic would highly determine the essential spreading patterns once the network structure is established. The results may shed some light on the in-depth understanding of the underlying dynamics of information transmission on real social networks.
Evaluating Spillover Effects in Network-Based Studies In the Presence of Missing Outcomes

TingFang Lee,Ashley L. Buchanan,Natallia Katenka,Laura Forastiere,M. Elizabeth Halloran,Georgios Nikolopoulos

DOI: https://doi.org/10.48550/arXiv.2304.09361

2023-04-19

Methodology

Abstract:Estimating causal effects in the presence of spillover among individuals embedded within a social network is often challenging with missing information. The spillover effect is the effect of an intervention if a participant is not exposed to the intervention themselves but is connected to intervention recipients in the network. In network-based studies, outcomes may be missing due to the administrative end of a study or participants being lost to follow-up due to study dropout, also known as censoring. We propose an inverse probability censoring weighted (IPCW) estimator, which is an extension of an IPW estimator for network-based observational studies to settings where the outcome is subject to possible censoring. We demonstrated that the proposed estimator was consistent and asymptotically normal. We also derived a closed-form estimator of the asymptotic variance estimator. We used the IPCW estimator to quantify the spillover effects in a network-based study of a nonrandomized intervention with censoring of the outcome. A simulation study was conducted to evaluate the finite-sample performance of the IPCW estimators. The simulation study demonstrated that the estimator performed well in finite samples when the sample size and number of connected subnetworks (components) were fairly large. We then employed the method to evaluate the spillover effects of community alerts on self-reported HIV risk behavior among people who inject drugs and their contacts in the Transmission Reduction Intervention Project (TRIP), 2013 to 2015, Athens, Greece. Community alerts were protective not only for the person who received the alert from the study but also among others in the network likely through information shared between participants. In this study, we found that the risk of HIV behavior was reduced by increasing the proportion of a participant's immediate contacts exposed to community alerts.
Behavioral Spillovers with Interdependent Institutions: An Experimental Study

Jenna Bednar,Yan Chen,Xiao Liu,Scott Page

DOI: https://doi.org/10.2139/ssrn.999437

2007-01-01

SSRN Electronic Journal

Abstract:We describe laboratory experiments that produce behavioral spillovers across strategic contexts. In these experiments subjects play two distinct games simulta- neously with dierent opponents. We find that behavior is highly context dependent: when games are paired in ensembles, play diers from the isolated controls. Behavior is also influenced by which other game composes the ensemble, and in predicted ways. These results suggest that people do not treat each strategic situation in isolation but instead construct heuristics that they apply across games. The results reject the hy- pothesis that subjects play games independently; instead the findings imply that the eect of a particular institution on behavior depends upon the full institutional con- text. The results have implications for any attempt to transport institutions across contexts, including developmental programs and constitutional designs.
Positive and negative spillover of pro-environmental behavior: An integrative review and theoretical framework

Heather Barnes Truelove,Amanda R. Carrico,Elke U. Weber,Kaitlin Toner Raimi,Michael P. Vandenbergh

DOI: https://doi.org/10.1016/j.gloenvcha.2014.09.004

IF: 11.16

2014-11-01

Global Environmental Change

Abstract:A recent surge of research has investigated the potential of pro-environmental behavior interventions to affect other pro-environmental behaviors not initially targeted by the intervention. The evidence evaluating these spillover effects has been mixed, with some studies finding evidence for positive spillover (i.e., one pro-environmental behavior increases the likelihood of performing additional pro-environmental behaviors) and others finding negative spillover (i.e., one pro-environmental behavior decreases the likelihood of additional pro-environmental behaviors). Different academic disciplines have investigated this question, employing different methodologies and arriving at divergent findings. This paper provides a unifying theoretical framework and uses the framework to review the existing research on pro-environmental behavior spillover. Our framework identifies different decision modes as competing mechanisms that drive adoption of initial pro-environmental behaviors, with different consequences for subsequent pro-environmental behaviors, leading to positive, negative, or no spillover. Attribution of the initial pro-environmental behavior to either an external motivator (e.g., a price signal) or internal motivator (e.g., self-identity) also matters. In addition, the characteristics of and similarity between initial and subsequent pro-environmental behaviors can be expected to moderate predicted spillover effects. We explore the implications of our model for policymakers and practitioners, and suggest key areas where future research on the topic would be most beneficial.
Behavioral Spillovers and Cognitive Load in Multiple Games: an Experimental Study

Jenna Bednar,Yan Chen,Tracy Xiao Liu,Scott E. Page

DOI: https://doi.org/10.1016/j.geb.2011.06.009

IF: 1.265

2012-01-01

Games and Economic Behavior

Abstract:We present evidence from laboratory experiments of behavioral spillovers and cognitive load that spread across strategic contexts. In the experiments, subjects play two distinct games simultaneously with different opponents. We find that the strategies chosen and the efficiency of outcomes in one game depends on the other game that the subject plays, and that play is altered in predictable directions. We develop a measure of behavioral variation in a normal form game, outcome entropy, and find that prevalent strategies in games with low outcome entropy are more likely to be used in the games with high outcome entropy, but not vice versa. Taken together, these findings suggest that people do not treat strategic situations in isolation, but may instead develop heuristics that they apply across games.
Estimating Spillovers from Sampled Connections

Kieran Marray

2024-10-23

Abstract:Empirical researchers often estimate spillover effects by fitting linear or non-linear regression models to sampled network data. Here, we show that common sampling schemes induce dependence between observed and unobserved spillovers. Due to this dependence, spillover estimates are biased, often upwards. We then show how researchers can construct unbiased estimates of spillover effects by rescaling using aggregate network statistics. Our results can be used to bound true effect sizes, determine robustness of estimates to missingness, and construct estimates when missingness depends on treatment. We apply our results to re-estimate the propagation of idiosyncratic shocks between US public firms, and peer effects amongst USAFA cadets.

General Economics

Spillover Effects in Online Field Experiments

Spillover Effects in Online Field Experiments: An Annotated Reading List

Detecting Spillover Effects: Design and Analysis of Multilevel Experiments

Is Attention Contagious? Estimating The Spillover Effect Of Investor Attention In Digital Networks

Online or offline? Spillover effect of customer-to-customer interaction in a multichannel background

Spillover effects in environmental behaviors, across time and context: a review and research agenda

Beyond the Targeted Customer: Spillover Effect Through Social Influence

Effects of online-to-offline spillovers on pricing and quality strategies of competing firms

Identification and estimation of spillover effects in randomized experiments

Programme evaluation and spillover effects

Spillover modes in multiplex games: double-edged effects on cooperation, and their coevolution

Online Field Experiments: a Selective Survey of Methods

Spillover Effect of Content Marketing in E-commerce Platform under the Fan Economy Era.

Beyond the Coverage of Information Spreading: Analytical and Empirical Evidence of Re-exposure in Large-scale Online Social Networks

Targeted Promotions and Cross-Category Spillover Effects

How Events Determine Spreading Patterns: Information Transmission Via Internal and External Influences on Social Networks

Evaluating Spillover Effects in Network-Based Studies In the Presence of Missing Outcomes

Behavioral Spillovers with Interdependent Institutions: An Experimental Study

Positive and negative spillover of pro-environmental behavior: An integrative review and theoretical framework

Behavioral Spillovers and Cognitive Load in Multiple Games: an Experimental Study

Estimating Spillovers from Sampled Connections