Domain-adapted Learning and Interpretability: DRL for Gas Trading

Yuanrong Wang,Yinsen Miao,Alexander CY Wong,Nikita P Granger,Christian Michler

2023-09-11

Abstract:Deep Reinforcement Learning (Deep RL) has been explored for a number of applications in finance and stock trading. In this paper, we present a practical implementation of Deep RL for trading natural gas futures contracts. The Sharpe Ratio obtained exceeds benchmarks given by trend following and mean reversion strategies as well as results reported in literature. Moreover, we propose a simple but effective ensemble learning scheme for trading, which significantly improves performance through enhanced model stability and robustness as well as lower turnover and hence lower transaction cost. We discuss the resulting Deep RL strategy in terms of model explainability, trading frequency and risk measures.

Trading and Market Microstructure

What problem does this paper attempt to address?

The paper primarily aims to address the issues encountered when applying Deep Reinforcement Learning (DRL) in natural gas futures trading, particularly how to improve the performance and interpretability of trading strategies. Specifically, the paper addresses the following key issues:

1. **Feature Selection and Importance Analysis**: By using the SHAP (SHapley Additive exPlanations) method to evaluate the impact of different features on model decisions, effective feature selection is conducted.

2. **Walk-Forward Training Scheme Selection**: Two different walk-forward training schemes—anchored window method and sliding window method—are compared to determine which scheme better adapts to market changes and improves the model's generalization ability.

3. **Enhancing Model Stability and Robustness**: A simple yet effective ensemble learning scheme based on threshold voting is proposed to stabilize the model's behavior, reduce trading frequency, and lower trading costs.

4. **Model Interpretability**: By conducting interpretability analysis of the model's output results, such as using SHAP plots and other visualization tools, the transparency of the model's decision-making process is improved.

5. **Role of Technical Indicators and Fundamental Data**: The study investigates the role of technical indicators (such as MACD, RSI, etc.) and fundamental data (such as demand, production, imports, inventory, etc.) in predicting natural gas price movements and how this information can be effectively utilized by the model.

Through the above research, the paper aims to develop a high-performance and easily understandable deep reinforcement learning model that can adapt to ever-changing market conditions for automated trading in the natural gas futures market.