Abstract:The transition of display ad exchanges from second-price auctions (SPA) to first-price auctions (FPA) has raised questions about its impact on revenue. Auction theory predicts the revenue equivalence between these two auction formats. However, display ad auctions are different from standard models in auction theory. First, automated bidding agents cannot easily derive equilibrium strategies in FPA because information regarding competitors is not readily available. Second, due to principal-agent problems, bidding agents typically maximize return-on-investment (ROI), not payoff. The literature on learning agents for real-time bidding is growing because of the practical relevance of this area; most research has found that learning agents do not converge to an equilibrium. Specifically, research on algorithmic collusion in display ad auctions has argued that FPA can induce symmetric Q-learning agents to tacitly collude, resulting in bids below equilibrium, leading to lower revenue compared to the SPA. Whether bids are in equilibrium cannot easily be determined from field data since the underlying values of bidders are unknown. In this paper, we draw on analytical modeling and numerical experiments and explore the convergence behavior of widespread online learning algorithms in both complete and incomplete information models. Contrary to prior results, we show that there are no systematic deviations from equilibrium behavior. We also explore the differences in revenue of the FPA and SPA, which have not been done for utility functions relevant to this domain, such as ROI. We show that learning algorithms also converge to equilibrium. Still, revenue equivalence does not hold, indicating that collusion may not be the explanation for lower revenue with FPA, and the change in auction format might have had substantial and non-obvious consequences for ad exchanges and advertisers.

Econometrics for Learning Agents

Bid Prediction in Repeated Auctions with Learning

Paying to Do Better: Games with Payments between Learning Agents

Infer Your Enemies and Know Yourself, Learning in Real-Time Bidding with Partially Observable Opponents

On the Convergence of Learning Algorithms in Bayesian Auction Games

Inference and auction design in online advertising

Using Reinforcement Learning to Validate Empirical Game-Theoretic Analysis: A Continuous Double Auction Study

From Behavioral Theories to Econometrics: Inferring Preferences of Human Agents from Data on Repeated Interactions

Computing Bayes Nash Equilibrium Strategies in Auction Games via Simultaneous Online Dual Averaging

Dynamic Incentive-Aware Learning: Robust Pricing in Contextual Auctions

Learning in Budgeted Auctions with Spacing Objectives

Equilibrium Learning in Combinatorial Auctions: Computing Approximate Bayesian Nash Equilibria via Pseudogradient Dynamics

A Game-Theoretic Analysis of the Empirical Revenue Maximization Algorithm with Endogenous Sampling.

Data-Driven Behaviour Estimation in Parametric Games

Using Multi-Agent Reinforcement Learning in Auction Simulations

Revenue in First- and Second-Price Display Advertising Auctions: Understanding Markets with Learning Agents

Learning Truthful, Efficient, and Welfare Maximizing Auction Rules

On Bayesian Epistemology of Myerson Auction.

Verifying Approximate Equilibrium in Auctions

Learning Equilibria of Simulation-Based Games

Equilibrium Computation in Multi-Stage Auctions and Contests