Abstract:The optimal stopping problem is a category of decision problems with a specific constrained configuration. It is relevant to various real-world applications such as finance and management. To solve the optimal stopping problem, state-of-the-art algorithms in dynamic programming, such as the least-squares Monte Carlo (LSMC), are employed. This type of algorithm relies on path simulations using only the last price of the underlying asset as a state representation. Also, the LSMC was thinking for option valuation where risk-neutral probabilities can be employed to account for uncertainty. However, the general optimal stopping problem goals may not fit the requirements of the LSMC showing auto-correlated prices. We employ a data-driven method that uses Monte Carlo simulation to train and test artificial neural networks (ANN) to solve the optimal stopping problem. Using ANN to solve decision problems is not entirely new. We propose a different architecture that uses convolutional neural networks (CNN) to deal with the dimensionality problem that arises when we transform the whole history of prices into a Markovian state. We present experiments that indicate that our proposed architecture improves results over the previous implementations under specific simulated time series function sets. Lastly, we employ our proposed method to compare the optimal exercise of the financial options problem with the LSMC algorithm. Our experiments show that our method can capture more accurate exercise opportunities when compared to the LSMC. We have outstandingly higher (above 974\% improvement) expected payoff from these exercise policies under the many Monte Carlo simulations that used the real-world return database on the out-of-sample (test) data.

Randomized Optimal Stopping Problem in Continuous Time and Reinforcement Learning Algorithm

Randomized Optimal Stopping Problem in Continuous time and Reinforcement Learning Algorithm

Exploratory Optimal Stopping: A Singular Control Formulation

Learning to Optimally Stop a Diffusion Process

Optimal Stopping via Randomized Neural Networks

Solving the optimal stopping problem with reinforcement learning: an application in financial option exercise

Robust optimal stopping with regime switching

A Nonparametric Algorithm for Optimal Stopping Based on Robust Optimization

On an Optimal Stopping Problem with a Discontinuous Reward

Deep Reinforcement Learning for Optimal Stopping with Application in Financial Engineering

Recursive Optimal Stopping with Poisson Stopping Constraints

Solving optimal stopping problems with Deep Q-Learning

Sequential Design for Optimal Stopping Problems

Optimal stopping problem under random horizon

Optimal Stopping under Model Ambiguity: a Time-Consistent Equilibrium Approach

The Optimal Stopping Problem under a Random Horizon

On the rates of convergence of simulation based optimization algorithms for optimal stopping problems

Control randomisation approach for policy gradient and application to reinforcement learning in optimal switching

Actively Learning Reinforcement Learning: A Stochastic Optimal Control Approach

Optimal Scheduling of Entropy Regularizer for Continuous-Time Linear-Quadratic Reinforcement Learning

Data-driven optimal stopping: A pure exploration analysis