Abstract:Dual control denotes a class of control problems where the parameters governing the system are imperfectly known. The challenge is to find the optimal balance between probing, i.e. exciting the system to understand it more, and caution, i.e. selecting conservative controls based on current knowledge to achieve the control objective. Dynamic programming techniques can achieve this optimal trade-off. However, while dynamic programming performs well with discrete state and time, it is not well-suited to problems with continuous time-frames or continuous or unbounded state spaces. Another limitation is that multidimensional states often cause the dynamic programming approaches to be intractable. In this paper, we investigate whether continuous-time optimal control tools could help circumvent these caveats whilst still achieving the probing-caution balance. We introduce a stylized problem where the state is governed by one of two differential equations. It is initially unknown which differential equation governs the system, so we must simultaneously determine the true differential equation and control the system to the desired state. We show how this problem can be transformed to apply optimal control tools, and compare the performance of this approach to a dynamic programming approach. Our results suggest that the optimal control algorithm rivals dynamic programming on small problems, achieving the right balance between aggressive and smoothly varying controls. In contrast to dynamic programming, the optimal control approach remains tractable when several states are to be controlled simultaneously.

An Alternative Dual Control Approach for Auto-Optimization in Uncertain Environments

Dual Control of Exploration and Exploitation for Auto-Optimisation Control with Active Learning

Dual Adaptive Control

Multistep Dual Control for Exploration and Exploitation in Autonomous Search With Convergence Guarantee

Adaptive dual control with online outlier detection for uncertain systems

Adaptive Reinforcement Learning Strategy-Based Sliding Mode Control of Uncertain Euler-Lagrange Systems with Prescribed Performance Guarantees: Autonomous Underwater Vehicles-Based Verification

Convergence Depth Control for Process System Optimization

Quasi-Newton Extremum Seeking Control Based on Incremental Recursive Identification.

Optimal control of discrete event systems under uncertain environment based on supervisory control theory and reinforcement learning

Extremum Seeking Nonlinear Regulator with Concurrent Uncertainties in Exosystems and Control Directions

Event-Triggered Robust Optimal Control for Nonlinear Systems With Uncertain Dynamics and Nonzero-equilibrium

Output information-based intermittent optimal control for continuous-time nonlinear systems with unmatched uncertainties via adaptive dynamic programming

Safe Linear-Quadratic Dual Control with Almost Sure Performance Guarantee

Global Output-Feedback Extremum Seeking Control with Source Seeking Experiments

Optimistic value-based optimal control problems with uncertain discrete-time noncausal systems

A continuous time formulation of stochastic dual control to avoid the curse of dimensionality

Prescribed Time Dual-Mode Extremum Seeking Control

Active Uncertainty Reduction for Human-Robot Interaction: An Implicit Dual Control Approach

A primal--dual algorithm as applied to optimal control problems

Robust Adaptive Extremum Seeking Control Without Persistence of Excitation: Theory to Experiment

Sequential learning and control: Targeted exploration for robust performance