Self-adaptive Pairs Trading Model Based on Reinforcement Learning Algorithm

Wenwei HU,Jianqiang HU,Zhan LI,Jianfeng ZHOU
DOI: https://doi.org/10.3969/j.issn.1672-0334.2017.02.012
2017-01-01
Journal of Management Science
Abstract:Pairs trading is one of the major statistical arbitrage trading strategies.However, its profit opportunity has become scarcer due to the improvement of the market efficiency.The traditional fixed parameter trading models are no longer sufficient for eternal profit maximization.The parameters of the trading models need not only to be optimized but also to be done so dynamically in an automatic manner.Therefore, it is necessary to develop a trading model of which parameters are dynamically optimized with artificial intelligence, as it may be of significance in improving the profitability and efficiency of trading models.A new type of statistical arbitrage trading model is proposed based on the reinforcement learning mode, improving the traditional cointegration trading strategy;Applying the Sarsa algorithm and ε-greedy strategy to the new model, the key parameters in the new trading model can self-adapt to reach the optimal values, instead of judging from professional experience or insisting on determined parameters just like the traditional strategy;A computer simulation is designed to run through the complete process of the new trading model including model parameters self-adapting adjustment, securities transaction, and trading performance evaluation.The trading simulation and empirical tests such as Johansen cointegration test, t-test, and Robustness test are conducted on four bond pairs that are composed of the top five bonds with the largest trading volumes in the mainland markets.The results show that the new model outperforms the traditional one in all aspects.It significantly enhances the profitability of the trading system while reducing the drawdown risks;It improves the efficiency of arbitrage trading as it reduces the number of transactions and thus transaction costs;It possesses ability to learn continuously so that it increases the accumulated return step by step and eventually converges to the highest level.The results also reveal that the cointegration trading strategy is efficient in the Chinese bond markets.
What problem does this paper attempt to address?