Measuring Consumer Sensitivity to Audio Advertising: A Long-Run Field Experiment on Pandora Internet Radio

Ali Goli,Jason Huang,David Reiley,Nickolai M. Riabov

2024-12-07

Abstract:A randomized experiment with almost 35 million Pandora listeners enables us to measure the sensitivity of consumers to advertising, an important topic of study in the era of ad-supported digital content provision. The experiment randomized listeners into nine treatment groups, each of which received a different level of audio advertising interrupting their music listening, with the highest treatment group receiving more than twice as many ads as the lowest treatment group. By maintaining consistent treatment assignment for 21 months, we measure long-run demand effects and find ad-load sensitivity three times greater than what we would have obtained from a month-long experiment. We show the negative impact on the number of hours listened, days listened, and probability of listening at all in the final month. Using an experimental design that separately varies the number of commercial interruptions per hour and the number of ads per commercial interruption, we find that listeners primarily respond to the total number of ads per hour, with a slight preference for more frequent but shorter ad breaks. Lastly, we find that increased ad load led to an increase in the number of paid ad-free subscriptions to Pandora. Importantly, we show that observational methods often lead to biased or even directionally incorrect estimates of these effects, highlighting the value of experimental data.

General Economics

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is: **How does consumers' sensitivity to advertisements affect their behavior on advertisement - supported digital content platforms?**

Specifically, this paper studied the responses of 35 million listeners on the Pandora Internet radio platform to different audio advertisement loads (ad load) through a large - scale, long - term randomized controlled experiment. The experimental design randomly assigned listeners to nine different ad load groups, with each group receiving a different number of advertisements. The highest ad load group received more than twice as many advertisements as the lowest group. The experiment lasted for 21 months to measure the long - term impact of ad load on user behavior.

### Main research questions:
1. **The impact of ad load on user behavior**: The paper explored how ad load affects users' music - listening duration, active days, and the probability of continuing to use Pandora.

2. **The impact of advertisement frequency and advertisement length**: By changing the number of advertisement interruptions per hour and the number of advertisements in each advertisement interruption respectively, it studied whether users prefer frequent but shorter advertisements or fewer but longer ones.

3. **The impact of ad load on paid subscriptions**: It studied whether an increase in ad load would lead more users to switch to the paid ad - free version of Pandora.

4. **Differences between experimental data and observational data**: It verified the advantages of experimental data over observational data and showed that the latter may lead to biased or directionally wrong results.

### Research methods:
- **Large - scale randomized controlled experiment**: The experiment involved 35 million users, divided into 9 different ad load groups plus a control group.
- **Long - term tracking**: The experiment lasted for 21 months to ensure capturing the long - term impact of ad load on user behavior.
- **Multi - dimensional analysis**: It not only focused on the total amount of advertisements but also analyzed the impacts of advertisement frequency and advertisement length separately.

### Key findings:
- **Long - term elasticity is greater than short - term elasticity**: In the long run, the impact of ad load on user behavior is three times that of the short - term impact.
- **Advertisement frequency is better than advertisement length**: Users prefer frequent but shorter advertisements.
- **An increase in ad load leads to an increase in paid subscriptions**: An increase in ad load significantly increases the likelihood of users purchasing the paid ad - free version.
- **Observational data are biased**: Traditional observational methods often overestimate or underestimate the impact of ad load on user behavior and even reach opposite conclusions.

Through these findings, the paper emphasizes the importance of experimental data in understanding the impact of ad load on user behavior and provides a basis for advertisement - supported digital content platforms to optimize their advertising strategies.