Abstract:Model-free continuous control for robot navigation tasks using Deep Reinforcement Learning (DRL) that relies on noisy policies for exploration is sensitive to the density of rewards. In practice, robots are usually deployed in cluttered environments, containing many obstacles and narrow passageways. Designing dense effective rewards is challenging, resulting in exploration issues during training. Such a problem becomes even more serious when tasks are described using temporal logic specifications. This work presents a deep policy gradient algorithm for controlling a robot with unknown dynamics operating in a cluttered environment when the task is specified as a Linear Temporal Logic (LTL) formula. To overcome the environmental challenge of exploration during training, we propose a novel path planning-guided reward scheme by integrating sampling-based methods to effectively complete goal-reaching missions. To facilitate LTL satisfaction, our approach decomposes the LTL mission into sub-goal-reaching tasks that are solved in a distributed manner. Our framework is shown to significantly improve performance (effectiveness, efficiency) and exploration of robots tasked with complex missions in large-scale cluttered environments. A video demonstration can be found on YouTube Channel: <a class="link-external link-https" href="https://youtu.be/yMh_NUNWxho" rel="external noopener nofollow">this https URL</a>.

Deep Reinforcement Learning with Temporal Logics

Directed Exploration in Reinforcement Learning from Linear Temporal Logic

Reinforcement Learning for Temporal Logic Control Synthesis with Probabilistic Satisfaction Guarantees

Modular Deep Reinforcement Learning with Temporal Logic Specifications

Mission-driven Exploration for Accelerated Deep Reinforcement Learning with Temporal Logic Task Specifications

LCRL: Certified Policy Synthesis via Logically-Constrained Reinforcement Learning

Certified Reinforcement Learning with Logic Guidance

DeepLTL: Learning to Efficiently Satisfy Complex LTL Specifications

Reinforcement learning under temporal logic constraints as a sequence modelling problem

Formal Policy Synthesis for Continuous-Space Systems via Reinforcement Learning

Reinforcement learning under temporal logic constraints as a sequence modeling problem

Reinforcement Learning with Temporal-Logic-Based Causal Diagrams

Modular Deep Reinforcement Learning for Continuous Motion Planning With Temporal Logic

Overcoming Exploration: Deep Reinforcement Learning for Continuous Control in Cluttered Environments from Temporal Logic Specifications

Control Synthesis from Linear Temporal Logic Specifications using Model-Free Reinforcement Learning

Reinforcement Learning Based Temporal Logic Control with Maximum Probabilistic Satisfaction

Deep Reinforcement Learning Under Signal Temporal Logic Constraints Using Lagrangian Relaxation

A Framework for Following Temporal Logic Instructions with Unknown Causal Dependencies

Eventual Discounting Temporal Logic Counterfactual Experience Replay

Model-Free Reinforcement Learning for Stochastic Games with Linear Temporal Logic Objectives

Safe Reinforcement Learning for Signal Temporal Logic Tasks Using Robust Control Barrier Functions