Abstract:Online control with non-stochastic disturbances and adversarially chosen convex cost functions, referred to as online non-stochastic control, has recently attracted increasing attention. We study online non-stochastic control with partial feedback, where learners can only access partially observed states and partially informed (bandit) costs. The problem setting arises naturally in real-world decision-making applications and strictly generalizes exceptional cases studied disparately by previous works. We propose the first online algorithm for this problem, with an $\tilde{O}(T^{3/4})$ regret competing with the best policy in hindsight, where $T$ denotes the time horizon and the $\tilde{O}(\cdot)$-notation omits the poly-logarithmic factors in $T$. To further enhance the algorithms' robustness to changing environments, we then design a novel method with a two-layer structure to optimize the dynamic regret, a more challenging measure that competes with time-varying policies. Our method is based on the online ensemble framework by treating the controller above as the base learner. On top of that, we design two different meta-combiners to simultaneously handle the unknown variation of environments and the memory issue arising from the online control. We prove that the two resulting algorithms enjoy $\tilde{O}(T^{3/4}(1+P_T)^{1/2})$ and $\tilde{O}(T^{3/4}(1+P_T)^{1/4}+T^{5/6})$ dynamic regret respectively, where $P_T$ measures the environmental non-stationarity. Our results are further extended to unknown transition matrices. Finally, empirical studies in both synthetic linear and simulated nonlinear tasks validate our method's effectiveness, thus supporting the theoretical findings.

Convergence Analysis for an Online Data-Driven Feedback Control Algorithm

Convergence Analysis for A Stochastic Maximum Principle Based Data Driven Feedback Control Algorithm

A nonlinear predictive control algorithm based on fuzzy online modeling and discrete optimization

An Online Algorithm for Solving Feedback Optimal Control Problems with Partial Observations

Data-Driven Method for Response Control of Nonlinear Random Dynamical Systems

Online Control with Adversarial Disturbance for Continuous-time Linear Systems

Online accelerated data‐driven learning for optimal feedback control of discrete‐time partially uncertain systems

Online Data-Driven Adaptive Control for Unknown Linear Time-Varying Systems

Near Optimal Neural Network-based Output Feedback Control of Affine Nonlinear Discrete-Time Systems

Online Consensus Control of Nonlinear Affine Systems From Disturbed Data

Convergence Analysis for a Cooperative Distributed Model Predictive Control Algorithm Based on Fixed Point Iteration.

Online Non-stochastic Control with Partial Feedback

Online Estimation and Adaptive Control for a Class of History Dependent Functional Differential Equations

Online Adaptive Optimal Control for Continuous-Time Nonlinear Systems with Completely Unknown Dynamics.

Adaptive Optimal Control with Guaranteed Convergence Rate for Continuous-Time Linear Systems with Completely Unknown Dynamics.

Convergence of machine learning methods for feedback control laws: averaged feedback learning scheme and data driven method

A Novel Data-Driven Controller Tuning Method For Improving Convergence Performance

Data-Driven Adversarial Online Control for Unknown Linear Systems

State and Input Constrained Output-Feedback Adaptive Optimal Control of Affine Nonlinear Systems

Online Primal-Dual Methods with Measurement Feedback for Time-Varying Convex Optimization

On-line data-driven control for uncertain systems based on greedy algorithm