Abstract:We consider the dynamics and the interactions of multiple reinforcement learning optimal execution trading agents interacting with a reactive Agent-Based Model (ABM) of a financial market in event time. The model represents a market ecology with 3-trophic levels represented by: optimal execution learning agents, minimally intelligent liquidity takers, and fast electronic liquidity providers. The optimal execution agent classes include buying and selling agents that can either use a combination of limit orders and market orders, or only trade using market orders. The reward function explicitly balances trade execution slippage against the penalty of not executing the order timeously. This work demonstrates how multiple competing learning agents impact a minimally intelligent market simulation as functions of the number of agents, the size of agents' initial orders, and the state spaces used for learning. We use phase space plots to examine the dynamics of the ABM, when various specifications of learning agents are included. Further, we examine whether the inclusion of optimal execution agents that can learn is able to produce dynamics with the same complexity as empirical data. We find that the inclusion of optimal execution agents changes the stylised facts produced by ABM to conform more with empirical data, and are a necessary inclusion for ABMs investigating market micro-structure. However, including execution agents to chartist-fundamentalist-noise ABMs is insufficient to recover the complexity observed in empirical data.

Reinforcement learning in market games

Stochastic Market Games

Reinforcement Learning in Non-Markov Market-Making

Reinforcement Learning in Economics and Finance

Using Reinforcement Learning to Validate Empirical Game-Theoretic Analysis: A Continuous Double Auction Study

Towards Multi-Agent Reinforcement Learning driven Over-The-Counter Market Simulations

Many learning agents interacting with an agent-based market model

Financial Trading as a Game: A Deep Reinforcement Learning Approach

Reinforcement Learning for Mean Field Game

A Linear Programming-Based Reinforcement Learning Mechanism for Incomplete-Information Games

Exploring Dominant Strategies in Iterated and Evolutionary Games: a Multi-Agent Reinforcement Learning Approach

Reinforcement Learning in High-frequency Market Making

Robust Risk-Sensitive Reinforcement Learning Agents for Trading Markets

Reinforcement Learning Approaches to Optimal Market Making

Reinforcement Learning for Market Making in a Multi-agent Dealer Market

Using Reinforcement Learning in the Algorithmic Trading Problem

Reinforcement Learning in Agent-Based Market Simulation: Unveiling Realistic Stylized Facts and Behavior

Deep Reinforcement Learning and Influenced Games

A memory-based spatial evolutionary game with the dynamic interaction between learners and profiteers

Counter-Factual Reinforcement Learning: How to Model Decision-Makers That Anticipate The Future

Paying to Do Better: Games with Payments between Learning Agents