A General Framework on Enhancing Portfolio Management with Reinforcement Learning

Yinheng Li,Junhao Wang,Yijie Cao
2023-10-27
Abstract:Portfolio management is the art and science in fiance that concerns continuous reallocation of funds and assets across financial instruments to meet the desired returns to risk profile. Deep reinforcement learning (RL) has gained increasing interest in portfolio management, where RL agents are trained base on financial data to optimize the asset reallocation process. Though there are prior efforts in trying to combine RL and portfolio management, previous works did not consider practical aspects such as transaction costs or short selling restrictions, limiting their applicability. To address these limitations, we propose a general RL framework for asset management that enables continuous asset weights, short selling and making decisions with relevant features. We compare the performance of three different RL algorithms: Policy Gradient with Actor-Critic (PGAC), Proximal Policy Optimization (PPO), and Evolution Strategies (ES) and demonstrate their advantages in a simulated environment with transaction costs. Our work aims to provide more options for utilizing RL frameworks in real-life asset management scenarios and can benefit further research in financial applications.
Portfolio Management,Machine Learning
What problem does this paper attempt to address?
The paper aims to address the optimization problem in financial portfolio management. Specifically, the authors focus on how to use deep reinforcement learning (RL) to improve asset allocation strategies to maximize returns and control risks. Compared to previous research, the main contributions of this paper are: 1. **Algorithm Comparison**: The paper compares three different reinforcement learning algorithms—Policy Gradient with Actor-Critic (PGAC), Proximal Policy Optimization (PPO), and Evolution Strategies (ES). These three algorithms were compared in a simulated environment, taking into account transaction costs. 2. **Continuous Weights and Short Selling**: Traditional research usually deals with discrete asset weight allocations or does not allow short selling operations, but this paper allows continuous asset weights and short selling, making its approach more applicable to real-world scenarios. 3. **Feature Input**: In addition to price information, the model also allows the use of other market features as input, thereby improving the rationality and accuracy of the decision-making process. Through these improvements, the paper hopes to provide more solutions for actual asset management and promote further research in the field of financial applications. Experimental results show that among the tested algorithms, the CNN-based PPO model performed the best, while ES, despite using a relatively simple neural network structure, also demonstrated good performance. Overall, all RL-based methods outperformed traditional rule-driven strategies in most cases.