Auto Uning of Price Prediction Models for High-Frequency Trading Via Reinforcement Learning
Weipeng Zhang,Ning Zhang,Junchi Yan,Guofu Li,Xiaokang Yang
DOI: https://doi.org/10.1016/j.patcog.2022.108543
IF: 8
2022-01-01
Pattern Recognition
Abstract:A B S T R A C T In this paper, we propose an online model optimization algorithm based on reinforcement learning for quantitative trading. The combination of prediction model and trading policy is the most commonly used framework in practical quantitative trading. Integrated with machine learning methods, this framework brings huge profits to quantified companies. In the framework, the prediction model is used to predict future trading price trend, and the trading policy is used to determine the price and number of orders. Even though, the shortcomings of machine learning models are obvious, mainly are, (1) Slow prediction speed. Huge human-craft features and model computing cost much time, which is ten times of pure trading policy without model. (2) Poor generalization. This kind of models can hardly adapt to market data in each period, because market traders will change time to time at micro level, thus the distribution of market data will change. But current model is trained on a long period dataset, it achieves best effect at average, but can not adapt to different market at each period. To address this problem, we propose a novel online model optimization algorithm. A light model library will be constructed. Each light model in this library corresponds to a different market distribution. By devising the appropriate reward function via inverse reinforcement learning algorithm, the algorithm can accurately estimate the profits of each model. Then the model can be selected automatically in real-time trading, so that the trading policies can automatically adapt to changes in trading market, overcoming previous shortcoming of manually updating model and slow prediction speed. Experimental results show that the proposed algorithm achieves stateof-the-art performance on China Commodity Futures Market Data. (c) 2022 Elsevier Ltd. All rights reserved.