Abstract:We consider the dynamics and the interactions of multiple reinforcement learning optimal execution trading agents interacting with a reactive Agent-Based Model (ABM) of a financial market in event time. The model represents a market ecology with 3-trophic levels represented by: optimal execution learning agents, minimally intelligent liquidity takers, and fast electronic liquidity providers. The optimal execution agent classes include buying and selling agents that can either use a combination of limit orders and market orders, or only trade using market orders. The reward function explicitly balances trade execution slippage against the penalty of not executing the order timeously. This work demonstrates how multiple competing learning agents impact a minimally intelligent market simulation as functions of the number of agents, the size of agents' initial orders, and the state spaces used for learning. We use phase space plots to examine the dynamics of the ABM, when various specifications of learning agents are included. Further, we examine whether the inclusion of optimal execution agents that can learn is able to produce dynamics with the same complexity as empirical data. We find that the inclusion of optimal execution agents changes the stylised facts produced by ABM to conform more with empirical data, and are a necessary inclusion for ABMs investigating market micro-structure. However, including execution agents to chartist-fundamentalist-noise ABMs is insufficient to recover the complexity observed in empirical data.

Multi-agent reinforcement learning in a realistic limit order book market simulation

Asynchronous Deep Double Duelling Q-Learning for Trading-Signal Execution in Limit Order Book Markets

Optimal Execution with Reinforcement Learning

Asynchronous Deep Double Dueling Q-learning for trading-signal execution in limit order book markets

Many learning agents interacting with an agent-based market model

Learning Multi-Agent Intention-Aware Communication for Optimal Multi-Order Execution in Finance

Neural Stochastic Agent-Based Limit Order Book Simulation: A Hybrid Methodology

Towards Multi-Agent Reinforcement Learning driven Over-The-Counter Market Simulations

Model-based Reinforcement Learning for Predictions and Control for Limit Order Books

Towards Generalizable Reinforcement Learning for Trade Execution

Market Making with Deep Reinforcement Learning from Limit Order Books

Modeling limit order trading with a continuous action policy for deep reinforcement learning

Optimizing Market Making using Multi-Agent Reinforcement Learning

Order book regulatory impact on stock market quality: a multi-agent reinforcement learning perspective

Multi-Agent Deep Reinforcement Learning for High-Frequency Multi-Market Making

Reinforcement Learning for Market Making in a Multi-agent Dealer Market

Deep Q-Learning Market Makers in a Multi-Agent Simulated Stock Market

Reinforcement Learning in Agent-Based Market Simulation: Unveiling Realistic Stylized Facts and Behavior

Towards a fully RL-based Market Simulator

Multi-Agent Deep Reinforcement Learning for Liquidation Strategy Analysis

Learn Continuously, Act Discretely: Hybrid Action-Space Reinforcement Learning For Optimal Execution