Stochastic maximum principle for optimal control problem with varying terminal time and non-convex control domain

Jin Shi,Shuzhen Yang

2024-09-04

Abstract:In this paper, we consider a varying terminal time structure for the stochastic optimal control problem under state constraints, in which the terminal time varies with the mean value of the state. In this new stochastic optimal control system, the control domain does not need to be convex and the diffusion coefficient contains the control variable. To overcome the difficulty in the proof of the related Pontryagin's stochastic maximum principle, we develop asymptotic first- and second-order adjoint equations for the varying terminal time, and then establish its variational equation. In the end, two examples are given to verify the main results of this study.

Optimization and Control

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to consider the situation where the terminal time changes with the state mean in the stochastic optimal control problem with state constraints. Specifically:

1. **Research Background**:
- In the classical stochastic optimal control problem, the terminal time is fixed.
- This paper introduces a new structure, in which the terminal time \(\tau_u\) is determined according to the mean of the state \(X_u(t)\):
\[
\tau_u=\inf\left\{t:E[\Phi(X_u(t))]\geq\alpha,t\in[0,T]\right\}\wedge T
\]
- Here, \(\Phi\) is a given function, and \(\alpha\) is a constant.

2. **Problem Description**:
- The objective is to minimize the following cost functional with a variable terminal time:
\[
J(u(\cdot)) = E\left[\int_0^{\tau_u}f(X_u(t),u(t))dt + g(X_u(\tau_u))\right]
\]
- where \(X_u(t)\) satisfies the following stochastic differential equation:
\[
dX_u(t)=b(X_u(t),u(t))dt+\sigma(X_u(t),u(t))dW(t)
\]

3. **Main Challenges**:
- The control domain \(U\) is not necessarily a convex set, and the diffusion coefficient \(\sigma\) contains the control variable \(u\).
- This makes it difficult to directly apply the traditional Pontryagin stochastic maximum principle.

4. **Solutions**:
- The paper overcomes these difficulties by introducing asymptotic first - order and second - order adjoint equations and establishes variational equations under variable terminal time.
- Finally, the author uses the "spike variation method" to establish the global stochastic maximum principle.

5. **Contributions**:
- This paper provides the global stochastic maximum principle for the stochastic optimal control problem with variable terminal time in the non - convex control domain.
- This result fills a gap in this field and provides a theoretical basis for subsequent research.

In summary, this paper aims to solve the maximum principle problem of the stochastic optimal control problem under the conditions of non - convex control domain and variable terminal time. By introducing new mathematical tools and methods, the author successfully derives the global stochastic maximum principle applicable to such problems.