Deep reinforcement learning for portfolio management of markets with a dynamic number of assets

Carlos Betancourt,Wen-Hui Chen

DOI: https://doi.org/10.1016/j.eswa.2020.114002

IF: 8.5

2021-02-01

Expert Systems with Applications

Abstract:<p>This work proposes a novel portfolio management method using deep reinforcement learning on markets with a dynamic number of assets. This problem is especially important in cryptocurrency markets, which already support the trading of hundreds of assets with new ones being added every month. A novel neural network architecture is proposed, which is trained using deep reinforcement learning. Our architecture considers all assets in the market, and automatically adapts when new ones are suddenly introduced, making our method more general and sample-efficient than previous methods. Further, transaction cost minimization is considered when formulating the problem. For this purpose, a novel algorithm to compute optimal transactions given a desired portfolio is integrated into the architecture. The proposed method was tested on a dataset of one of the largest cryptocurrency markets in the world, outperforming state-of-the-art methods, achieving average daily returns of over 24%.</p>

computer science, artificial intelligence,engineering, electrical & electronic,operations research & management science

What problem does this paper attempt to address?

The paper aims to address the problem of portfolio management in markets with a dynamic number of assets, with a particular focus on the cryptocurrency market. Traditional approaches assume a fixed number of assets in the market, which is not applicable to the cryptocurrency market as new cryptocurrencies are continuously being added. To solve this problem, the authors propose a new method based on deep reinforcement learning.

Specifically, the main contributions of the paper include:

1. **Framework for Adapting to a Dynamic Number of Assets**:
- A new neural network architecture is proposed that can handle a dynamically changing number of assets. This method is more general and sample-efficient compared to existing methods.
- The new architecture can automatically adapt to the addition of new assets without the need for additional training or memory.

2. **Minimizing Transaction Costs**:
- Transaction costs are considered in the problem modeling, and a new algorithm is proposed to calculate the optimal transaction volume.
- By inputting the current portfolio vector into the network output layer, the method penalizes unheld assets, thereby reducing the number of transactions.
- A linear programming problem is proposed to solve the optimal trading strategy for a given target portfolio proportion.

The paper conducts experiments on a large cryptocurrency market dataset, and the results show that the proposed method outperforms existing benchmark methods under different test settings, achieving an average daily return rate of over 24%. Additionally, the paper provides a detailed description of the design and implementation process of the method and discusses the experimental results.