Abstract:Market making (MM) has attracted significant attention in financial trading owing to its essential function in ensuring market liquidity. With strong capabilities in sequential decision-making, Reinforcement Learning (RL) technology has achieved remarkable success in quantitative trading. Nonetheless, most existing RL-based MM methods focus on optimizing single-price level strategies which fail at frequent order cancellations and loss of queue priority. Strategies involving multiple price levels align better with actual trading scenarios. However, given the complexity that multi-price level strategies involves a comprehensive trading action space, the challenge of effectively training profitable RL agents for MM persists. Inspired by the efficient workflow of professional human market makers, we propose Imitative Market Maker (IMM), a novel RL framework leveraging both knowledge from suboptimal signal-based experts and direct policy interactions to develop multi-price level MM strategies efficiently. The framework start with introducing effective state and action representations adept at encoding information about multi-price level orders. Furthermore, IMM integrates a representation learning unit capable of capturing both short- and long-term market trends to mitigate adverse selection risk. Subsequently, IMM formulates an expert strategy based on signals and trains the agent through the integration of RL and imitation learning techniques, leading to efficient learning. Extensive experimental results on four real-world market datasets demonstrate that IMM outperforms current RL-based market making strategies in terms of several financial criteria. The findings of the ablation study substantiate the effectiveness of the model components.

Deep Reinforcement Learning for Market Making

Deep Reinforcement Learning for High-Frequency Market Making

Multi-Agent Deep Reinforcement Learning for High-Frequency Multi-Market Making

Market Making Strategy Optimization via Deep Reinforcement Learning

Performance of Deep Reinforcement Learning for High Frequency Market Making on Actual Tick Data.

Reinforcement Learning for Market Making in a Multi-agent Dealer Market

Market Making with Deep Reinforcement Learning from Limit Order Books

Deep Q-Learning Market Makers in a Multi-Agent Simulated Stock Market

Reinforcement Learning Approaches to Optimal Market Making

Reinforcement Learning in Non-Markov Market-Making

Deep Reinforcement Learning for Market Making Under a Hawkes Process-Based Limit Order Book Model

Market Making under Order Stacking Framework: A Deep Reinforcement Learning Approach

Extending Deep Reinforcement Learning Frameworks in Cryptocurrency Market Making

Optimizing Market Making using Multi-Agent Reinforcement Learning

Automated market maker inventory management with deep reinforcement learning

A reinforcement learning approach to improve the performance of the Avellaneda-Stoikov market-making algorithm

Market making and incentives design in the presence of a dark pool: a deep reinforcement learning approach

Deep Hawkes process for high-frequency market making

IMM: An Imitative Reinforcement Learning Approach with Predictive Representation Learning for Automatic Market Making