Abstract:Order execution is an extremely important problem in the financial domain, and recently, more and more researchers have tried to employ reinforcement learning (RL) techniques to solve this challenging problem. There are a lot of difficulties for conventional RL methods to tackle the order execution problem, such as the large action space including price and quantity, and the long-horizon property. As naturally order execution is composed of a low-frequency volume scheduling stage and a high-frequency order placement stage, most existing RL-based order execution methods treat these stages as two distinct tasks and offer a partial solution by addressing either one individually. However, the current literature fails to model the non-negligible mutual influence between these two tasks, leading to impractical order execution solutions. To address these limitations, we propose a novel automatic order execution approach based on the hierarchical RL framework (OEHRL), which jointly learns the policies for volume scheduling and order placement. OEHRL first extracts the state embeddings at both the macro and micro levels with a sequential variational auto-encoder model. Based on the effective embeddings, OEHRL generates a hindsight expert dataset, which is used to train a hierarchical order execution policy. In the hierarchical structure, the high-level policy is in charge of the target volume and the low-level learns to determine the prices for a series of the allocated sub-orders from the high level. These two levels collaborate seamlessly and contribute to the optimal order execution policy. Extensive experiment results on 200 stocks across the US and China A-share markets validate the effectiveness of the proposed approach.

Optimal Execution Using Reinforcement Learning

Optimal Execution with Reinforcement Learning

Learning Multi-Agent Intention-Aware Communication for Optimal Multi-Order Execution in Finance

Double Deep Q-Learning for Optimal Execution

Learn Continuously, Act Discretely: Hybrid Action-Space Reinforcement Learning For Optimal Execution

Optimal Exploration Algorithm of Multi-Agent Reinforcement Learning Methods (Student Abstract)

Joint Learning of Volume Scheduling and Order Placement Policies for Optimal Order Execution

A reinforcement learning approach to optimal execution

Multi-agent reinforcement learning in a realistic limit order book market simulation

Towards Generalizable Reinforcement Learning for Trade Execution

Universal Trading for Order Execution with Oracle Policy Distillation

Optimal order placement in limit order markets

Optimal Execution Considering Trading Signal and Execution Risk Simultaneously

Optimal Order Execution subject to Reservation Strategies under Execution Risk

Practical Application of Deep Reinforcement Learning to Optimal Trade Execution

Risk-Sensitive Compact Decision Trees for Autonomous Execution in Presence of Simulated Market Response

Optimal Order Execution Using Hidden Orders

Deep Stock Trading: A Hierarchical Reinforcement Learning Framework for Portfolio Optimization and Order Execution

Optimal Trade Execution in Illiquid Markets

Optimal Execution under Incomplete Information