Abstract:We consider the dynamics and the interactions of multiple reinforcement learning optimal execution trading agents interacting with a reactive Agent-Based Model (ABM) of a financial market in event time. The model represents a market ecology with 3-trophic levels represented by: optimal execution learning agents, minimally intelligent liquidity takers, and fast electronic liquidity providers. The optimal execution agent classes include buying and selling agents that can either use a combination of limit orders and market orders, or only trade using market orders. The reward function explicitly balances trade execution slippage against the penalty of not executing the order timeously. This work demonstrates how multiple competing learning agents impact a minimally intelligent market simulation as functions of the number of agents, the size of agents' initial orders, and the state spaces used for learning. We use phase space plots to examine the dynamics of the ABM, when various specifications of learning agents are included. Further, we examine whether the inclusion of optimal execution agents that can learn is able to produce dynamics with the same complexity as empirical data. We find that the inclusion of optimal execution agents changes the stylised facts produced by ABM to conform more with empirical data, and are a necessary inclusion for ABMs investigating market micro-structure. However, including execution agents to chartist-fundamentalist-noise ABMs is insufficient to recover the complexity observed in empirical data.

Learning to Classify and Imitate Trading Agents in Continuous Double Auction Markets

Learning to simulate realistic limit order book markets from data as a World Agent

On the Importance of Opponent Modeling in Auction Markets

Scalable Agent-Based Modeling for Complex Financial Market Simulations

A simple learning agent interacting with an agent-based market model

Multi-agent reinforcement learning in a realistic limit order book market simulation

Towards Multi-Agent Reinforcement Learning driven Over-The-Counter Market Simulations

Learning who is in the market from time series: market participant discovery through adversarial calibration of multi-agent simulators

TradingAgents: Multi-Agents LLM Financial Trading Framework

Many learning agents interacting with an agent-based market model

Infer Your Enemies and Know Yourself, Learning in Real-Time Bidding with Partially Observable Opponents

Modelling Opaque Bilateral Market Dynamics in Financial Trading: Insights from a Multi-Agent Simulation Study

Use of Markov Chains to Design an Agent Bidding Strategy for Continuous Double Auctions

Asynchronous Deep Double Duelling Q-Learning for Trading-Signal Execution in Limit Order Book Markets

Neural Stochastic Agent-Based Limit Order Book Simulation: A Hybrid Methodology

A Network Simulation of OTC Markets with Multiple Agents

Simulation and estimation of an agent-based market-model with a matching engine

Asynchronous Deep Double Dueling Q-learning for trading-signal execution in limit order book markets

Statistical theory of the continuous double auction

Double Auctions: Formalization and Automated Checkers

Stock market microstructure inference via multi-agent reinforcement learning