Abstract:Algorithmic trading is a crucial yet challenging task in the financial domain, where trading decisions are made sequentially from milliseconds to days based on the historical price movements and trading frequency. To model such a sequential decision making process in the dynamic financial markets, Deep Reinforcement Learning (DRL) based methods have been applied and demonstrated their success in finding trading strategies that achieve profitable returns. However, the financial markets are complex imperfect information games with high-level of noise and uncertainties which usually make the exploration policy of DRL less effective. In this paper, we propose an end-to-end DRL method that explores solutions on the whole graph via a probabilistic dynamic programming algorithm. Specifically, we separate the state into environment state and position state, and model the position state transition as a directed acyclic graph. To obtain reliable gradients for model training, we adopt a probabilistic dynamic programming algorithm to explore solutions over the whole graph instead of sampling a path. By avoiding the sampling procedure, we propose an efficient training algorithm and overcome the efficiency problem in most existing DRL methods. Furthermore, our method is compatible with most recurrent neural network architecture, which makes our method easy to implement and very effective in practice. Extensive experiments have been conducted on two real-world stock datasets. Experimental results demonstrate that our method can generate stable trading strategies for both high-frequency and low-frequency trading, significantly outperforming the baseline DRL methods on annualized return and Sharpe ratio.

Stock Market Trading Agent Using On-Policy Reinforcement Learning Algorithms

Agent Performing Autonomous Stock Trading under Good and Bad Situations

Evaluation of Deep Reinforcement Learning Based Stock Trading.

Application of multilayer perceptron to deep reinforcement learning for stock market trading and analysis

Deep Reinforcement Learning Robots for Algorithmic Trading: Considering Stock Market Conditions and U.S. Interest Rates

Using Reinforcement Learning in the Algorithmic Trading Problem

Practical Deep Reinforcement Learning Approach for Stock Trading

Optimizing Trading Strategies in Quantitative Markets using Multi-Agent Reinforcement Learning

Deep Reinforcement Learning Approach for Trading Automation in The Stock Market

Adversarial Deep Reinforcement Learning in Portfolio Management

Deep Reinforcement Learning for Stock Portfolio Optimization

Robust Risk-Sensitive Reinforcement Learning Agents for Trading Markets

Capturing Financial markets to apply Deep Reinforcement Learning

Deep Robust Reinforcement Learning for Practical Algorithmic Trading

Deep Reinforcement Learning for Stock Trading with Behavioral Finance Strategy.

Model-based Deep Reinforcement Learning for Dynamic Portfolio Optimization

Deep Reinforcement Learning Approach to Portfolio Optimization in the Australian Stock Market

Trading in Fast-Changing Markets with Meta-Reinforcement Learning

Optimal Action Space Search: an Effective Deep Reinforcement Learning Method for Algorithmic Trading

Portfolio Management using Deep Reinforcement Learning

Automatic Financial Trading Agent for Low-risk Portfolio Management using Deep Reinforcement Learning