Output Feedback Adaptive Optimal Control of Affine Nonlinear systems with a Linear Measurement Model

Tochukwu Elijah Ogri,S. M. Nahid Mahmud,Zachary I. Bell,Rushikesh Kamalapurkar

DOI: https://doi.org/10.48550/arXiv.2210.06637

2023-04-03

Abstract:Real-world control applications in complex and uncertain environments require adaptability to handle model uncertainties and robustness against disturbances. This paper presents an online, output-feedback, critic-only, model-based reinforcement learning architecture that simultaneously learns and implements an optimal controller while maintaining stability during the learning phase. Using multiplier matrices, a convenient way to search for observer gains is designed along with a controller that learns from simulated experience to ensure stability and convergence of trajectories of the closed-loop system to a neighborhood of the origin. Local uniform ultimate boundedness of the trajectories is established using a Lyapunov-based analysis and demonstrated through simulation results, under mild excitation conditions.

Systems and Control

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to achieve real - time control of nonlinear systems in complex and uncertain environments, especially for those systems with high model uncertainty and the need for robustness against external disturbances. Specifically, the paper focuses on how to design an online, output - feedback - based, critic - only model - based reinforcement learning architecture that can simultaneously learn and implement the optimal controller during the learning process while maintaining system stability. The main challenges mentioned in the paper include:

1. **Dealing with model uncertainty**: In practical applications, system models are often uncertain, which makes traditional control methods difficult to work effectively. The method proposed in the paper aims to overcome these uncertainties through adaptive learning.

2. **Ensuring stability during the learning process**: During the learning process, ensuring system stability and performance is a key issue. The paper achieves this by designing an observer to estimate the system state and combining it with the model - based predictive control (MBRL) framework, ensuring that system stability can be maintained even during the learning stage.

3. **Solving the control problem of nonlinear systems with partially constrained inputs**: The paper targets nonlinear systems with partially constrained inputs, which are very common in practical applications, such as in robot control, aerospace, etc. The method proposed in the paper can effectively handle the control problems of such systems while ensuring system performance and stability.

4. **Optimizing control performance**: In addition to maintaining system stability, the paper also aims to improve system performance by optimizing the control strategy. By minimizing a given cost function, the method proposed in the paper can find a near - optimal control strategy.

In summary, the core objective of this paper is to develop an online adaptive optimal control method that can work effectively in complex and uncertain environments, especially suitable for nonlinear systems with partially constrained inputs, while ensuring stability during the learning process and the final control performance.