Abstract:Optimal stopping is a fundamental problem in optimization that has found applications in risk management, finance, economics, and recently in the fields of computer science. We extend the standard framework to a multi-agent setting, named multi-agent optimal stopping (MAOS), where a group of agents cooperatively solves finite-space, discrete-time optimal stopping problems. Solving the finite-agent case is computationally prohibitive when the number of agents is very large, so this work studies the mean field optimal stopping (MFOS) problem, obtained as the number of agents approaches infinity. We prove that MFOS provides a good approximate solution to MAOS. We also prove a dynamic programming principle (DPP), based on the theory of mean field control. We then propose two deep learning methods: one simulates full trajectories to learn optimal decisions, whereas the other leverages DPP with backward induction; both methods train neural networks for the optimal stopping decisions. We demonstrate the effectiveness of these approaches through numerical experiments on 6 different problems in spatial dimension up to 300. To the best of our knowledge, this is the first work to study MFOS in finite space and discrete time, and to propose efficient and scalable computational methods for this type of problem.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper aims to solve the Multi - Agent Optimal Stopping (MAOS) problem, especially the Mean Field Optimal Stopping (MFOS) problem in finite space and discrete time. Specifically: 1. **Background and Motivation**: - Optimal Stopping (OS) is a fundamental problem in the field of optimization and is widely used in risk management, finance, economics, and computer science, etc. - When multiple agents are involved, the MAOS problem becomes more complex, especially when the number of agents is very large, it becomes computationally intractable. 2. **Problem Description**: - **Multi - Agent Optimal Stopping (MAOS)**: Multiple agents need to stop a series of dynamic systems at different time points to minimize the common cost function. - **Mean Field Optimal Stopping (MFOS)**: When the number of agents tends to infinity, the MAOS problem is approximately solved through the mean - field theory. This approximation method can significantly reduce the computational complexity. 3. **Main Contributions**: - **Theoretical Aspects**: - It is proved that MFOS can provide approximately optimal stopping decisions for the MAOS problem of N agents, and the convergence rate is \( O\left(\frac{1}{\sqrt{N}}\right) \) (Theorem 2.2). - By interpreting MFOS as a special mean - field control problem, a dynamic programming principle (DPP) is proved (Theorem 3.1). - **Computational Aspects**: - Two deep - learning - based methods are proposed to solve the MFOS problem: - **Direct Approach (DA)**: Learn the optimal decision by simulating the complete trajectory. - **Dynamic Programming Approach (DP)**: Use DPP and backward induction to train the neural network for optimal stopping decisions. - Numerical experiments are carried out in 6 different problems to verify the effectiveness of these methods, including problems with a spatial dimension as high as 300. 4. **Innovative Points**: - This is the first time to study the MFOS problem in discrete time and finite space and propose efficient and scalable computational methods. - The connection between MFOS and the mean - field control problem is revealed, providing a new research perspective. In summary, by introducing the mean - field theory and deep - learning methods, this paper solves the computational problems of large - scale multi - agent optimal stopping problems and lays a theoretical foundation for future research.

Deep Learning Algorithms for Mean Field Optimal Stopping in Finite Space and Discrete Time

A deep primal-dual BSDE method for optimal stopping problems

Deep Penalty Methods: A Class of Deep Learning Algorithms for Solving High Dimensional Optimal Stopping Problems

Solving the optimal stopping problem with reinforcement learning: an application in financial option exercise

Fictitious Play via Finite Differences for Mean Field Games with Optimal Stopping

Deep Learning Methods for Mean Field Control Problems With Delay

Deep Learning for Mean Field Games with non-separable Hamiltonians

Deep learning for ranking response surfaces with applications to optimal stopping problems

Deep Reinforcement Learning for Optimal Stopping with Application in Financial Engineering

A hybrid deep learning method for finite-horizon mean-field game problems

Deep Learning for Population-Dependent Controls in Mean Field Control Problems with Common Noise

Deep learning for solving initial path optimization of mean-field systems with memory

Recent Developments in Machine Learning Methods for Stochastic Control and Games

Deep Learning for Mean Field Games and Mean Field Control with Applications to Finance

A Machine Learning Algorithm for Finite-Horizon Stochastic Control Problems in Economics

Time-inconsistent mean-field stopping problems: A regularized equilibrium approach

Optimal Stopping via Randomized Neural Networks

Deep neural network expressivity for optimal stopping problems

Convergence of the Backward Deep BSDE Method with Applications to Optimal Stopping Problems

A Mean-Field Optimal Control Formulation of Deep Learning

A Class of Mean-Field Games with Optimal Stopping and Its Inverse Problem