Developing A Multi-Agent and Self-Adaptive Framework with Deep Reinforcement Learning for Dynamic Portfolio Risk Management

Zhenglong Li,Vincent Tam,Kwan L. Yeung

2024-09-10

Abstract:Deep or reinforcement learning (RL) approaches have been adapted as reactive agents to quickly learn and respond with new investment strategies for portfolio management under the highly turbulent financial market environments in recent years. In many cases, due to the very complex correlations among various financial sectors, and the fluctuating trends in different financial markets, a deep or reinforcement learning based agent can be biased in maximising the total returns of the newly formulated investment portfolio while neglecting its potential risks under the turmoil of various market conditions in the global or regional sectors. Accordingly, a multi-agent and self-adaptive framework namely the MASA is proposed in which a sophisticated multi-agent reinforcement learning (RL) approach is adopted through two cooperating and reactive agents to carefully and dynamically balance the trade-off between the overall portfolio returns and their potential risks. Besides, a very flexible and proactive agent as the market observer is integrated into the MASA framework to provide some additional information on the estimated market trends as valuable feedbacks for multi-agent RL approach to quickly adapt to the ever-changing market conditions. The obtained empirical results clearly reveal the potential strengths of our proposed MASA framework based on the multi-agent RL approach against many well-known RL-based approaches on the challenging data sets of the CSI 300, Dow Jones Industrial Average and S&P 500 indexes over the past 10 years. More importantly, our proposed MASA framework shed lights on many possible directions for future investigation.

Portfolio Management,Machine Learning

What problem does this paper attempt to address?

The paper aims to address the issue of dynamic risk management in financial portfolio management. Specifically, it points out that in highly volatile financial market environments, traditional deep learning or reinforcement learning methods may overlook potential risks when optimizing portfolio returns. To overcome this limitation, the paper proposes a Multi-Agent Self-Adaptive (MASA) framework that utilizes two cooperative agents to balance the trade-off between overall portfolio returns and potential risks. Additionally, a market observer agent is introduced to provide extra information about market trends, helping the system quickly adapt to changing market conditions.

Summary as follows:
- **Problem**: Traditional reinforcement learning methods overlook potential risks when optimizing portfolio returns.
- **Solution**: Propose the MASA framework, which achieves dynamic risk management through two cooperative agents (one based on the TD3 algorithm and one based on a constraint solver) and a market observer agent.
- **Objective**: Achieve a balance between portfolio returns and risks in highly volatile financial markets.