Abstract:Reinforcement learning (RL) has been applied to financial portfolio management in recent years. Current studies mostly focus on profit accumulation without much consideration of risk. Some risk‐return balanced studies extract features from price and volume data only, which is highly correlated and missing representation of risk features. To tackle these problems, we propose a weight control unit (WCU) to effectively manage the position of portfolio management in different market statuses. A loss penalty term is also designed in the reward function to prevent sharp drawdown during trading. Moreover, stock spatial interrelation representing the correlation between two different stocks is captured by a graph convolution network based on fundamental data. Temporal interrelation is also captured by a temporal convolutional network based on new factors designed with price and volume data. Both spatial and temporal interrelation work for better feature extraction from historical data and also make the model more interpretable. Finally, a deep deterministic policy gradient actor–critic RL is applied to explore optimal policy in portfolio management. We conduct our approach in a challenging non‐short‐selling market, and the experiment results show that our method outperforms the state‐of‐the‐art methods in both profit and risk criteria. Specifically, with 6.72% improvement on an annualized rate of return, 7.72% decrease in maximum drawdown, and a better annualized Sharpe ratio of 0.112. Also, the loss penalty and WCU provide new aspects for future work in risk control.

Continuous-time optimal investment with portfolio constraints: a reinforcement learning approach

Continuous‐time mean–variance portfolio selection: A reinforcement learning framework

Convergence Rate of Primal-Dual Approach to Constrained Reinforcement Learning with Softmax Policy

Learning Merton's Strategies in an Incomplete Market: Recursive Entropy Regularization and Biased Gaussian Exploration

Reinforcement Learning for Continuous-Time Mean-Variance Portfolio Selection in a Regime-Switching Market

Deep Reinforcement Learning for Stock Portfolio Optimization

Continual portfolio selection in dynamic environments via incremental reinforcement learning

Fundamental Limits of Reinforcement Learning in Environment with Endogeneous and Exogeneous Uncertainty

A General Framework on Enhancing Portfolio Management with Reinforcement Learning

Mean--Variance Portfolio Selection by Continuous-Time Reinforcement Learning: Algorithms, Regret Analysis, and Empirical Study

On optimal tracking portfolio in incomplete markets: The reinforcement learning approach

From deterministic to stochastic: an interpretable stochastic model-free reinforcement learning framework for portfolio optimization

Exploratory mean-variance portfolio selection with Choquet regularizers

Uncertainty-Aware Reinforcement Learning for Portfolio Optimization

A Deep Reinforcement Learning Approach for Portfolio Management in Non‐Short‐Selling Market

Model-based Deep Reinforcement Learning for Dynamic Portfolio Optimization

Bridging the gap between Markowitz planning and deep reinforcement learning

Reinforcement Learning for a Discrete-Time Linear-Quadratic Control Problem with an Application

Behavioral Portfolio Selection in Continuous Time

Evaluation of Deep Reinforcement Learning Algorithms for Portfolio Optimisation

Model Based Reinforcement Learning with Non-Gaussian Environment Dynamics and Its Application to Portfolio Optimization