Deep reinforcement learning with positional context for intraday trading

Sven Goluža,Tomislav Kovačević,Tessa Bauman,Zvonko Kostanjčar

DOI: https://doi.org/10.1007/s12530-024-09593-6

2024-06-12

Abstract:Deep reinforcement learning (DRL) is a well-suited approach to financial decision-making, where an agent makes decisions based on its trading strategy developed from market observations. Existing DRL intraday trading strategies mainly use price-based features to construct the state space. They neglect the contextual information related to the position of the strategy, which is an important aspect given the sequential nature of intraday trading. In this study, we propose a novel DRL model for intraday trading that introduces positional features encapsulating the contextual information into its sparse state space. The model is evaluated over an extended period of almost a decade and across various assets including commodities and foreign exchange securities, taking transaction costs into account. The results show a notable performance in terms of profitability and risk-adjusted metrics. The feature importance results show that each feature incorporating contextual information contributes to the overall performance of the model. Additionally, through an exploration of the agent's intraday trading activity, we unveil patterns that substantiate the effectiveness of our proposed model.

Trading and Market Microstructure

What problem does this paper attempt to address?

The paper attempts to address the issue that existing Deep Reinforcement Learning (DRL) models in intraday trading primarily rely on price features to construct the state space, neglecting contextual information related to the strategy's position. This neglect limits the model's performance in handling intraday trading tasks, as intraday trading is sequential, and the position information of the strategy is crucial for decision-making. For example, evaluating the remaining time before the end of the trading day to avoid overnight risk, or adjusting the trading strategy based on the current positions held. To solve this problem, the paper proposes a new DRL model that introduces position features, which contain the position information of the strategy relative to specific time points, thereby enriching the state space. In this way, the model can better understand and utilize contextual information during the trading process, improving the quality of decisions. Additionally, the paper extensively evaluates the proposed model, including its performance over nearly 10 years and across various assets (such as commodities and forex securities), considering the impact of transaction costs. The experimental results show that the model performs excellently in terms of profitability and risk-adjusted metrics, and each feature containing contextual information contributes to the overall performance of the model.

Deep reinforcement learning with positional context for intraday trading

Deep reinforcement learning with positional context for intraday trading

Deep reinforcement learning on a multi-asset environment for trading

DeepScalper: A Risk-Aware Reinforcement Learning Framework to Capture Fleeting Intraday Trading Opportunities

Deep Reinforcement Learning Approach for Trading Automation in The Stock Market

Deep Reinforcement Learning For Trading - A Critical Survey

From Bandits Model to Deep Deterministic Policy Gradient, Reinforcement Learning with Contextual Information

Deep Reinforcement Trading with Predictable Returns

Deep Reinforcement Learning in Quantitative Algorithmic Trading: A Review

A Primer on Deep Reinforcement Learning for Finance

Deep Reinforcement Learning for Active High Frequency Trading

Deep Reinforcement Learning Strategies in Finance: Insights into Asset Holding, Trading Behavior, and Purchase Diversity

Time-driven Feature-Aware Jointly Deep Reinforcement Learning for Financial Signal Representation and Algorithmic Trading

FinRL: Deep Reinforcement Learning Framework to Automate Trading in Quantitative Finance

Learning financial asset-specific trading rules via deep reinforcement learning

Deep Reinforcement Learning for Stock Trading with Behavioral Finance Strategy.

Deep Reinforcement Learning Based Optimization and Risk Control of Trading Strategies

A synchronous deep reinforcement learning model for automated multi-stock trading

An Application of Deep Reinforcement Learning to Algorithmic Trading

A novel deep reinforcement learning framework with BiLSTM-Attention networks for algorithmic trading

Deep differentiable reinforcement learning and optimal trading