Abstract:Algorithmic trading is a crucial yet challenging task in the financial domain, where trading decisions are made sequentially from milliseconds to days based on the historical price movements and trading frequency. To model such a sequential decision making process in the dynamic financial markets, Deep Reinforcement Learning (DRL) based methods have been applied and demonstrated their success in finding trading strategies that achieve profitable returns. However, the financial markets are complex imperfect information games with high-level of noise and uncertainties which usually make the exploration policy of DRL less effective. In this paper, we propose an end-to-end DRL method that explores solutions on the whole graph via a probabilistic dynamic programming algorithm. Specifically, we separate the state into environment state and position state, and model the position state transition as a directed acyclic graph. To obtain reliable gradients for model training, we adopt a probabilistic dynamic programming algorithm to explore solutions over the whole graph instead of sampling a path. By avoiding the sampling procedure, we propose an efficient training algorithm and overcome the efficiency problem in most existing DRL methods. Furthermore, our method is compatible with most recurrent neural network architecture, which makes our method easy to implement and very effective in practice. Extensive experiments have been conducted on two real-world stock datasets. Experimental results demonstrate that our method can generate stable trading strategies for both high-frequency and low-frequency trading, significantly outperforming the baseline DRL methods on annualized return and Sharpe ratio.

Deep Deterministic Portfolio Optimization

Deep differentiable reinforcement learning and optimal trading

Deep Reinforcement Learning for Stock Portfolio Optimization

Model-based Deep Reinforcement Learning for Dynamic Portfolio Optimization

High-Dimensional Stock Portfolio Trading with Deep Reinforcement Learning

Deep Reinforcement Trading with Predictable Returns

Optimal Action Space Search: an Effective Deep Reinforcement Learning Method for Algorithmic Trading

Dynamic Optimization of Portfolio Allocation Using Deep Reinforcement Learning

Portfolio Management using Deep Reinforcement Learning

Adversarial Deep Reinforcement Learning in Portfolio Management

Optimal Action Space Search

Evaluation of Deep Reinforcement Learning Algorithms for Portfolio Optimisation

Bridging the gap between Markowitz planning and deep reinforcement learning

Deep Reinforcement Learning Approach to Portfolio Optimization in the Australian Stock Market

Deep Robust Reinforcement Learning for Practical Algorithmic Trading

Deep Reinforcement Learning Strategies in Finance: Insights into Asset Holding, Trading Behavior, and Purchase Diversity

Optimizing Automated Trading Systems with Deep Reinforcement Learning

Deep reinforcement learning on a multi-asset environment for trading

Deep Reinforcement Learning for Online Optimal Execution Strategies

Practical Deep Reinforcement Learning Approach for Stock Trading

An Application of Deep Reinforcement Learning to Algorithmic Trading