Abstract:We propose two new optimistic planning algorithms for nonlinear hybrid-input systems, in which the input has both a continuous and a discrete component, and the discrete component must respect a dwell-time constraint. Both algorithms select sets of input sequences for refinement at each step, along with a continuous or discrete step to refine (split). The dwell-time constraint means that the discrete splits must keep the discrete mode constant if the required dwell-time is not yet reached. Convergence rate guarantees are provided for both algorithms, which show the dependency between the near-optimality of the sequence returned and the computational budget. The rates depend on a novel complexity measure of the dwell-time constrained problem. We present simulation results for two problems, an adaptive-quantization networked control system and a model for the COVID pandemic.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to achieve approximate optimal control in nonlinear systems with mixed inputs (i.e., containing both continuous and discrete inputs simultaneously) and where the discrete inputs are subject to minimum dwell - time constraints. Specifically, the paper proposes two new optimistic planning algorithms (OPHIS ∆ and SOPHIS ∆), which are able to select the set of input sequences to be refined at each step and decide whether to perform continuous - input or discrete - input refinement. The minimum dwell - time constraint on discrete inputs means that the discrete mode must remain unchanged until the required minimum dwell - time is reached.
### Problem Background
In many practical applications, such as robotics, industrial multi - tank systems, or the automotive industry, mixed - input systems are very common. The discrete inputs in these systems are usually subject to minimum dwell - time constraints to prevent overly rapid switching, which may be due to physical limitations or for the purpose of improving performance. However, existing methods usually do not consider such constraints or are only applicable to linear dynamic systems.
### Research Objectives
The main objectives of the paper are:
1. **Propose New Algorithms**: Design two new optimistic planning algorithms, OPHIS ∆ and SOPHIS ∆, for handling the approximate optimal control problem of mixed - input systems with minimum dwell - time constraints.
2. **Convergence Analysis**: Provide convergence rate guarantees for these two algorithms and analyze the relationship between the approximate optimal solution and the computational budget.
3. **Complexity Measurement**: Introduce a new complexity measurement to describe the complexity of problems with minimum dwell - time constraints.
4. **Practical Applications**: Verify the effectiveness of the algorithms through simulation results of two practical problems, namely the adaptive - quantization network control system and the COVID - 19 epidemic spread model.
### Main Contributions
1. **Algorithm Design**: Proposed two algorithms, OPHIS ∆ and SOPHIS ∆, which are able to select the set to be refined at each step and decide whether to perform continuous - input or discrete - input refinement.
2. **Convergence Analysis**: Provided convergence rate analysis for these two algorithms, showing the dependence between the approximate optimal solution and the computational budget.
3. **Complexity Measurement**: Introduced a new complexity measurement to describe the complexity of problems with minimum dwell - time constraints.
4. **Practical Applications**: Verified the effectiveness of the algorithms through simulation results of two practical problems, namely the adaptive - quantization network control system and the COVID - 19 epidemic spread model.
### Mathematical Model
The mathematical models used in the paper are as follows:
- **System Model**: Discrete - time nonlinear mixed - input system
\[
x_{k + 1}=f(x_k, u_k),\quad u_k = [c_k, d_k]^T
\]
where \(x\in X\subseteq\mathbb{R}^m\) is the state, \(u\in U\) is the input, containing a continuous action \(c_k\in\mathbb{R}\) and a discrete mode \(d_k\in\{0, 1,\ldots, p\}\).
- **Reward Function**: Immediate performance (negative cost) function
\[
r_{k + 1}=\rho(x_k, u_k)
\]
- **Infinite - Horizon Discounted Value**:
\[
v(u^{\infty})=\sum_{k = 0}^{\infty}\gamma^k\rho(x_k, u_k)
\]
where \(\gamma\in(0, 1)\) is the discount factor.
### Simulation Results
The paper verifies the effectiveness of the algorithms through simulation results of two practical problems:
1. **Adaptive - Quantization Network Control System**: Applied to the inverted pendulum system, the control commands are transmitted through the adaptive - quantization network configuration, reducing network usage while maintaining good control performance.
2. **COVID - 19 Epidemic Spread Model**: Applied to the epidemic spread model, the epidemic spread is controlled by optimizing vaccination strategies and isolation levels.
### Conclusion
The OPHIS ∆ and SOPHIS ∆ algorithms proposed in the paper can effectively solve the approximate optimal control problem of mixed - input systems with minimum dwell - time constraints and in practical applications.