Abstract:In a reinforcement learning (RL) framework, we study the exploratory version of the continuous time expected utility (EU) maximization problem with a portfolio constraint that includes widely-used financial regulations such as short-selling constraints and borrowing prohibition. The optimal feedback policy of the exploratory unconstrained classical EU problem is shown to be Gaussian. In the case where the portfolio weight is constrained to a given interval, the corresponding constrained optimal exploratory policy follows a truncated Gaussian distribution. We verify that the closed form optimal solution obtained for logarithmic utility and quadratic utility for both unconstrained and constrained situations converge to the non-exploratory expected utility counterpart when the exploration weight goes to zero. Finally, we establish a policy improvement theorem and devise an implementable reinforcement learning algorithm by casting the optimal problem in a martingale framework. Our numerical examples show that exploration leads to an optimal wealth process that is more dispersedly distributed with heavier tail compared to that of the case without exploration. This effect becomes less significant as the exploration parameter is smaller. Moreover, the numerical implementation also confirms the intuitive understanding that a broader domain of investment opportunities necessitates a higher exploration cost. Notably, when subjected to both short-selling and money borrowing constraints, the exploration cost becomes negligible compared to the unconstrained case.

A mean reversion selling rule with constraints

Pairs Trading: An Optimal Selling Rule with Constraints

Minimax perfect stopping rules for selling an asset near its ultimate maximum

An explicit solution for an optimal stopping/optimal control problem which models an asset sale

On penalized goal-reaching probability minimization under borrowing and short-selling constraints

Mean Reversion Trading with Sequential Deadlines and Transaction Costs

Optimal Trend Following Trading Rules

Mean–variance Portfolio Selection under No-Shorting Rules: A BSDE Approach

Optimal mean-variance reinsurance and investment strategy with constraints in a non-Markovian regime-switching model

Optimal Liquidation in a Mean-reverting Portfolio

Optimal Execution with Multiplicative Price Impact and Incomplete Information on the Return

Continuous-time optimal investment with portfolio constraints: a reinforcement learning approach

Optimal Mean Reversion Trading with Transaction Costs and Stop-Loss Exit

Optimal Investment and Excess of Loss Reinsurance with Short-Selling Constraint

Optimal Consumption and Sale Strategies for a Risk Averse Agent

Optimal pricing to minimize maximum regret with limited demand information

The right time to sell a stock whose price is driven by Markovian noise

Approximately optimal trade execution strategies under fast mean-reversion

A Mean-Field Game of Market Entry: Portfolio Liquidation with Trading Constraints

A Near Optimal Portfolio Rule for the Life Cycle Merton Problem

Optimal High Frequency Trading with limit and market orders