Abstract:Online control with non-stochastic disturbances and adversarially chosen convex cost functions, referred to as online non-stochastic control, has recently attracted increasing attention. We study online non-stochastic control with partial feedback, where learners can only access partially observed states and partially informed (bandit) costs. The problem setting arises naturally in real-world decision-making applications and strictly generalizes exceptional cases studied disparately by previous works. We propose the first online algorithm for this problem, with an $\tilde{O}(T^{3/4})$ regret competing with the best policy in hindsight, where $T$ denotes the time horizon and the $\tilde{O}(\cdot)$-notation omits the poly-logarithmic factors in $T$. To further enhance the algorithms' robustness to changing environments, we then design a novel method with a two-layer structure to optimize the dynamic regret, a more challenging measure that competes with time-varying policies. Our method is based on the online ensemble framework by treating the controller above as the base learner. On top of that, we design two different meta-combiners to simultaneously handle the unknown variation of environments and the memory issue arising from the online control. We prove that the two resulting algorithms enjoy $\tilde{O}(T^{3/4}(1+P_T)^{1/2})$ and $\tilde{O}(T^{3/4}(1+P_T)^{1/4}+T^{5/6})$ dynamic regret respectively, where $P_T$ measures the environmental non-stationarity. Our results are further extended to unknown transition matrices. Finally, empirical studies in both synthetic linear and simulated nonlinear tasks validate our method's effectiveness, thus supporting the theoretical findings.

Learning to Schedule Tasks with Deadline and Throughput Constraints.

Online Task Scheduling and Termination with Throughput Constraint

Online Lazy Bureaucrat Scheduling with a Machine Deadline

Learning to Schedule Online Tasks with Bandit Feedback

Developing Real-Time Scheduling Policy by Deep Reinforcement Learning

Learning-based Scheduling for Information Gathering with QoS Constraints

Survey of Weakly-Hard Real Time Schedule Theory and Its Application

Optimal differentiated threshold characterization for multi-task stochastic deadline scheduling with queuing

Reinforcement Learning for Online Dispatching Policy in Real-Time Train Timetable Rescheduling

Reinforcement Learning Based Online Scheduling of Multiple Workflows in Edge Environment

Single-Machine Scheduling with Simultaneous Learning Effects and Delivery Times

Offline Learning-Based Multi-User Delay-Constrained Scheduling

Asymptotically Optimal Lagrangian Priority Policy for Deadline Scheduling With Processing Rate Limits

Minimizing Mean Weighted Tardiness in Unrelated Parallel Machine Scheduling with Reinforcement Learning

A Two-Agent Single-Machine Scheduling Problem with a Time-Based Learning Effect.

Throughput-Optimal Scheduling for Multi-Hop Networked Transportation Systems With Switch-Over Delay

Learning to Schedule DAG Tasks

Neural Feedback Scheduling of Real-Time Control Tasks

Timely-Throughput Optimal Scheduling with Prediction.

Online Non-stochastic Control with Partial Feedback

High-speed Train Timetabling Based on Reinforcement Learning.