Abstract:Reinforcement learning (RL) techniques have shown great success in many challenging quantitative trading tasks, such as portfolio management and algorithmic trading. Especially, intraday trading is one of the most profitable and risky tasks because of the intraday behaviors of the financial market that reflect billions of rapidly fluctuating capitals. However, a vast majority of existing RL methods focus on the relatively low frequency trading scenarios (e.g., day-level) and fail to capture the fleeting intraday investment opportunities due to two major challenges: 1) how to effectively train profitable RL agents for intraday investment decision-making, which involves high-dimensional fine-grained action space; 2) how to learn meaningful multi-modality market representation to understand the intraday behaviors of the financial market at tick-level. Motivated by the efficient workflow of professional human intraday traders, we propose DeepScalper, a deep reinforcement learning framework for intraday trading to tackle the above challenges. Specifically, DeepScalper includes four components: 1) a dueling Q-network with action branching to deal with the large action space of intraday trading for efficient RL optimization; 2) a novel reward function with a hindsight bonus to encourage RL agents making trading decisions with a long-term horizon of the entire trading day; 3) an encoder-decoder architecture to learn multi-modality temporal market embedding, which incorporates both macro-level and micro-level market information; 4) a risk-aware auxiliary task to maintain a striking balance between maximizing profit and minimizing risk. Through extensive experiments on real-world market data spanning over three years on six financial futures, we demonstrate that DeepScalper significantly outperforms many state-of-the-art baselines in terms of four financial criteria.

Exploiting Risk-Aversion and Size-dependent fees in FX Trading with Fitted Natural Actor-Critic

Integrating Heuristics and Learning in a Computational Architecture for Cognitive Trading

Deep LSTM with Reinforcement Learning Layer for Financial Trend Prediction in FX High Frequency Trading Systems

Deep reinforcement learning on a multi-asset environment for trading

Evaluation of FOREX trading Strategies based in Random Forest and Support Vector Machines

Robust Risk-Sensitive Reinforcement Learning Agents for Trading Markets

Deep Reinforcement Trading with Predictable Returns

A Novel Anti-Risk Method for Portfolio Trading Using Deep Reinforcement Learning

Adaptive Quantitative Trading: An Imitative Deep Reinforcement Learning Approach

Adaptive Predictive Portfolio Management Agent

Combining Deep Learning on Order Books with Reinforcement Learning for Profitable Trading

Financial Trading as a Game: A Deep Reinforcement Learning Approach

A Deep Reinforcement Learning Approach for Trading Optimization in the Forex Market with Multi-Agent Asynchronous Distribution

Learning to Trade with Deep Actor Critic Methods

Deep Reinforcement Learning for Quantitative Trading

DeepScalper: A Risk-Aware Reinforcement Learning Framework to Capture Fleeting Intraday Trading Opportunities

Online Trading Models with Deep Reinforcement Learning in the Forex Market Considering Transaction Costs

Automatic Financial Trading Agent for Low-risk Portfolio Management using Deep Reinforcement Learning

Mesoscale impact of trader psychology on stock markets: a multi-agent AI approach

Risk-Averse Trust Region Optimization for Reward-Volatility Reduction

Mesoscale effects of trader learning behaviors in financial markets: A multi-agent reinforcement learning study