Abstract:Reinforcement learning algorithms discover policies that are lauded for their high efficiency, but don't necessarily guarantee safety. We introduce a new approach that provides the best of both worlds: learning optimal policies while enforcing the system to comply with certain model to keep the learning dependable. To this end, we propose Timed Differential Dynamic Logic to express the system properties. Our main insight is to convert the properties to runtime monitors, and use them to monitor whether the system is correctly modeled. We choose the optimal polices only if the reality matches the model, or we will abandon efficiency and instead to choose a policy that guides the agent to a modeled portion of the state space. We also propose Dependable Mixed Control (DMC) algorithm to implement a framework for application. Finally, the effectiveness of our approach is validated through a case study on Communication-Based Autonomous Control (CBAC).

Dependable Reinforcement Learning Via Timed Differential Dynamic Logic.