Using Multi-Agent Reinforcement Learning in Auction Simulations

Medet Kanmaz,Elif Surer

DOI: https://doi.org/10.48550/arXiv.2004.02764

2020-04-06

Abstract:Game theory has been developed by scientists as a theory of strategic interaction among players who are supposed to be perfectly rational. These strategic interactions might have been presented in an auction, a business negotiation, a chess game, or even in a political conflict aroused between different agents. In this study, the strategic (rational) agents created by reinforcement learning algorithms are supposed to be bidder agents in various types of auction mechanisms such as British Auction, Sealed Bid Auction, and Vickrey Auction designs. Next, the equilibrium points determined by the agents are compared with the outcomes of the Nash equilibrium points for these environments. The bidding strategy of the agents is analyzed in terms of individual rationality, truthfulness (strategy-proof), and computational efficiency. The results show that using a multi-agent reinforcement learning strategy improves the outcomes of the auction simulations.

Computer Science and Game Theory,Multiagent Systems

What problem does this paper attempt to address?

The problem that this paper attempts to solve is whether agents created through Multi - Agent Reinforcement Learning (MARL) can make decisions as rational agents in different auction mechanisms. Specifically, the researchers designed a reinforcement - learning environment in which agents participate as bidders in various types of auctions, such as English auctions, sealed - bid auctions, and Vickrey auctions, and receive positive or negative rewards according to the results of their actions. Each agent has a "private" evaluation value for the auctioned product, which is unknown to other agents. The action set of agents includes "bid" and "stop bidding". When the auction ends, the winner's reward is the difference between the auction transaction price and the agent's private evaluation value; while the loser will receive a negative reward representing the auction cost. The main objective of the paper is to analyze the performance of these strategic (rational) agents created by reinforcement - learning algorithms in different auction mechanisms, especially whether their equilibrium points can match the Nash equilibrium points. In addition, the paper also explores the characteristics of agents' bidding strategies in terms of individual rationality, truthfulness, and computational efficiency. The research results show that using multi - agent reinforcement - learning strategies can improve the effect of auction simulations, making the agents' behavior more in line with theoretical expectations, that is, reaching the Nash equilibrium under a given auction mechanism.

Using Multi-Agent Reinforcement Learning in Auction Simulations

Learning to Bid Long-Term: Multi-Agent Reinforcement Learning with Long-Term and Sparse Reward in Repeated Auction Games

Using Reinforcement Learning to Validate Empirical Game-Theoretic Analysis: A Continuous Double Auction Study

Towards Multi-Agent Reinforcement Learning driven Over-The-Counter Market Simulations

Understanding Iterative Combinatorial Auction Designs via Multi-Agent Reinforcement Learning

Artificial Intelligence for Multi-Unit Auction design

Approximating Auction Equilibria with Reinforcement Learning

Learning and Calibrating Heterogeneous Bounded Rational Market Behaviour with Multi-Agent Reinforcement Learning

Put Your Money Where Your Mouth Is: Evaluating Strategic Planning and Execution of LLM Agents in an Auction Arena

Strategic bidding in freight transport using deep reinforcement learning

Applying Opponent Modeling for Automatic Bidding in Online Repeated Auctions

Dynamic Pricing by Multiagent Reinforcement Learning

Equilibrium Computation in Multi-Stage Auctions and Contests

Use of Markov Chains to Design an Agent Bidding Strategy for Continuous Double Auctions

Econometrics for Learning Agents

Learning Best Response Policies in Dynamic Auctions via Deep Reinforcement Learning

Computing Bayes Nash Equilibrium Strategies in Auction Games via Simultaneous Online Dual Averaging

Deep Reinforcement Learning for Sequential Combinatorial Auctions

Reinforcement Learning for Market Making in a Multi-agent Dealer Market

Real-Time Bidding with Multi-Agent Reinforcement Learning in Display Advertising