Abstract:Model-free continuous control for robot navigation tasks using Deep Reinforcement Learning (DRL) that relies on noisy policies for exploration is sensitive to the density of rewards. In practice, robots are usually deployed in cluttered environments, containing many obstacles and narrow passageways. Designing dense effective rewards is challenging, resulting in exploration issues during training. Such a problem becomes even more serious when tasks are described using temporal logic specifications. This work presents a deep policy gradient algorithm for controlling a robot with unknown dynamics operating in a cluttered environment when the task is specified as a Linear Temporal Logic (LTL) formula. To overcome the environmental challenge of exploration during training, we propose a novel path planning-guided reward scheme by integrating sampling-based methods to effectively complete goal-reaching missions. To facilitate LTL satisfaction, our approach decomposes the LTL mission into sub-goal-reaching tasks that are solved in a distributed manner. Our framework is shown to significantly improve performance (effectiveness, efficiency) and exploration of robots tasked with complex missions in large-scale cluttered environments. A video demonstration can be found on YouTube Channel: <a class="link-external link-https" href="https://youtu.be/yMh_NUNWxho" rel="external noopener nofollow">this https URL</a>.

Reinforcement learning with temporal logic rewards

Reinforcement learning under temporal logic constraints as a sequence modelling problem

Reinforcement learning under temporal logic constraints as a sequence modeling problem

Hierarchical Temporal Logic Guided Reinforcement Learning

Deep Reinforcement Learning with Temporal Logics

Model-based Reinforcement Learning from Signal Temporal Logic Specifications

A Policy Search Method For Temporal Logic Specified Reinforcement Learning Tasks

Directed Exploration in Reinforcement Learning from Linear Temporal Logic

Adaptive Reward Design for Reinforcement Learning in Complex Robotic Tasks

Reinforcement Learning with Temporal-Logic-Based Causal Diagrams

Safe Reinforcement Learning for Signal Temporal Logic Tasks Using Robust Control Barrier Functions

Reinforcement Learning for Temporal Logic Control Synthesis with Probabilistic Satisfaction Guarantees

Mission-driven Exploration for Accelerated Deep Reinforcement Learning with Temporal Logic Task Specifications

Automata Guided Reinforcement Learning With Demonstrations

Exploiting Hybrid Policy in Reinforcement Learning for Interpretable Temporal Logic Manipulation

Tractable Reinforcement Learning of Signal Temporal Logic Objectives

Reinforcement Learning Based Temporal Logic Control with Maximum Probabilistic Satisfaction

A Framework for Following Temporal Logic Instructions with Unknown Causal Dependencies

Model-Free Reinforcement Learning for Stochastic Games with Linear Temporal Logic Objectives

Overcoming Exploration: Deep Reinforcement Learning for Continuous Control in Cluttered Environments from Temporal Logic Specifications

Certified Reinforcement Learning with Logic Guidance