Abstract:Financial portfolio management is the process of constant redistribution of a fund into different financial products. This paper presents a financial-model-free Reinforcement Learning framework to provide a deep machine learning solution to the portfolio management problem. The framework consists of the Ensemble of Identical Independent Evaluators (EIIE) topology, a Portfolio-Vector Memory (PVM), an Online Stochastic Batch Learning (OSBL) scheme, and a fully exploiting and explicit reward function. This framework is realized in three instants in this work with a Convolutional Neural Network (CNN), a basic Recurrent Neural Network (RNN), and a Long Short-Term Memory (LSTM). They are, along with a number of recently reviewed or published portfolio-selection strategies, examined in three back-test experiments with a trading period of 30 minutes in a cryptocurrency market. Cryptocurrencies are electronic and decentralized alternatives to government-issued money, with Bitcoin as the best-known example of a cryptocurrency. All three instances of the framework monopolize the top three positions in all experiments, outdistancing other compared trading algorithms. Although with a high commission rate of 0.25% in the backtests, the framework is able to achieve at least 4-fold returns in 50 days.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the decision - making problem in financial portfolio management. Specifically, the author proposes a framework based on deep reinforcement learning (DRL) to optimize the dynamic re - allocation of financial portfolios in order to maximize returns while controlling risks. ### Problem Background Traditional portfolio management methods rely on pre - constructed financial market models, and the effectiveness of these models may vary in different markets. In addition, some traditional methods require manual parameter setting or logical layers, and it is difficult to fully adapt to different market environments. Although price - prediction algorithms can predict asset price changes, their performance highly depends on the accuracy of prediction, and price prediction itself is not a market action. When it is transformed into actual trading behavior, an additional logical layer is required, which limits its flexibility and adaptability. ### Core Contributions of the Paper To solve the above problems, this paper proposes a deep reinforcement learning framework without a financial model, aiming to optimize portfolio management by directly learning market actions. The main features of this framework include: 1. **Ensemble of Identical Independent Evaluators (EIIE) Topology**: Each IIE is a neural network used to evaluate the historical performance of an asset and predict its future growth potential. 2. **Portfolio - Vector Memory (PVM)**: Records the portfolio weights in each period so as to consider the impact of trading costs on wealth. 3. **Online Stochastic Batch Learning (OSBL) Scheme**: Supports pre - trading training and online training. 4. **Explicit Reward Function**: Uses the average of periodic logarithmic returns as a reward to guide the model to be optimized in the direction of gradient ascent. ### Experimental Verification To verify the effectiveness and profitability of this framework, the author conducted three sets of back - testing experiments in the cryptocurrency market (such as Bitcoin, etc.), and the trading cycle of each experiment was 30 minutes. The results show that the models based on three different types of IIEs, namely CNN, RNN, and LSTM, significantly outperformed other existing investment strategies in these three experiments, achieving at least a 4 - fold return rate within 50 days, even with a relatively high trading fee (0.25%). ### Summary By introducing the deep reinforcement learning framework, the paper provides a brand - new portfolio management method that does not rely on specific financial models, demonstrating its superior performance in the complex and volatile cryptocurrency market. This framework can not only adapt to different market environments but also effectively handle practical problems such as trading costs.

A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem

A Deep Reinforcement Learning Framework For Financial Portfolio Management

Cryptocurrency Portfolio Management with Deep Reinforcement Learning

Portfolio management based on a reinforcement learning framework

Deep reinforcement learning for portfolio management of markets with a dynamic number of assets

Online portfolio management via deep reinforcement learning with high-frequency data

Deep Stock Trading: A Hierarchical Reinforcement Learning Framework for Portfolio Optimization and Order Execution

A General Framework on Enhancing Portfolio Management with Reinforcement Learning

A Deep Reinforcement Learning Approach for Portfolio Management in Non‐Short‐Selling Market

Online Optimal Investment Portfolio Model Based on Deep Reinforcement Learning

Multiagent-based deep reinforcement learning framework for multi-asset adaptive trading and portfolio management

Deep Reinforcement Learning for Asset Allocation in US Equities

High-Dimensional Stock Portfolio Trading with Deep Reinforcement Learning

Deep Graph Convolutional Reinforcement Learning for Financial Portfolio Management -- DeepPocket

Model-based Deep Reinforcement Learning for Dynamic Portfolio Optimization

Developing An Attention-Based Ensemble Learning Framework for Financial Portfolio Optimisation

Developing A Multi-Agent and Self-Adaptive Framework with Deep Reinforcement Learning for Dynamic Portfolio Risk Management

DeepTrader: A Deep Reinforcement Learning Approach for Risk-Return Balanced Portfolio Management with Market Conditions Embedding

A Novel Experts Advice Aggregation Framework Using Deep Reinforcement Learning for Portfolio Management

Reinforcement Learning Portfolio Manager Framework with Monte Carlo Simulation

CAD: Clustering And Deep Reinforcement Learning Based Multi-Period Portfolio Management Strategy