Abstract:Our research investigates the challenges Deep Reinforcement Learning (DRL) faces in complex, Partially Observable Markov Decision Processes (POMDP) such as autonomous driving (AD), and proposes a solution for vision-based navigation in these environments. Partial observability reduces RL performance significantly, and this can be mitigated by augmenting sensor information and data fusion to reflect a more Markovian environment. However, this necessitates an increasingly complex perception module, whose training via RL is complicated due to inherent limitations. As the neural network architecture becomes more complex, the reward function's effectiveness as an error signal diminishes since the only source of supervision is the reward, which is often noisy, sparse, and delayed. Task-irrelevant elements in images, such as the sky or certain objects, pose additional complexities. Our research adopts an offline-trained encoder to leverage large video datasets through self-supervised learning to learn generalizable representations. Then, we train a head network on top of these representations through DRL to learn to control an ego vehicle in the CARLA AD simulator. This study presents a broad investigation of the impact of different learning schemes for offline-training of encoders on the performance of DRL agents in challenging AD tasks. Furthermore, we show that the features learned by watching BDD100K driving videos can be directly transferred to achieve lane following and collision avoidance in CARLA simulator, in a zero-shot learning fashion. Finally, we explore the impact of various architectural decisions for the RL networks to utilize the transferred representations efficiently. Therefore, in this work, we introduce and validate an optimal way for obtaining suitable representations of the environment, and transferring them to RL networks.

Uncertainty-Aware Model-Based Offline Reinforcement Learning for Automated Driving

Uncertainty-Aware Model-Based Reinforcement Learning with Application to Autonomous Driving

Uncertainty-Aware Model-Based Reinforcement Learning: Methodology and Application in Autonomous Driving

Identify, Estimate and Bound the Uncertainty of Reinforcement Learning for Autonomous Driving

Offline Reinforcement Learning for Autonomous Driving with Safety and Exploration Enhancement

Uncertainty-Aware Decision-Making for Autonomous Driving at Uncontrolled Intersections

Tactical Decision-Making in Autonomous Driving by Reinforcement Learning with Uncertainty Estimation

Towards Safe Decision-Making for Autonomous Vehicles at Unsignalized Intersections

Uncertainty Sequence Modeling Approach for Safe and Effective Autonomous Driving

Addressing Inherent Uncertainty: Risk-Sensitive Behavior Generation for Automated Driving using Distributional Reinforcement Learning

Uncertainty-aware Distributional Offline Reinforcement Learning

Uncertainty-Aware Decision Transformer for Stochastic Driving Environments

Belief State Separated Reinforcement Learning for Autonomous Vehicle Decision Making under Uncertainty.

An Examination of Offline-Trained Encoders in Vision-Based Deep Reinforcement Learning for Autonomous Driving

Towards Robust Decision-Making for Autonomous Driving on Highway

Integrating Deep Reinforcement Learning with Model-based Path Planners for Automated Driving

UAC: Offline Reinforcement Learning with Uncertain Action Constraint

How to Learn from Risk: Explicit Risk-Utility Reinforcement Learning for Efficient and Safe Driving Strategies

A Simple Unified Uncertainty-Guided Framework for Offline-to-Online Reinforcement Learning

Adaptive Decision Making at the Intersection for Autonomous Vehicles Based on Skill Discovery

Roadside Units Assisted Localized Automated Vehicle Maneuvering: An Offline Reinforcement Learning Approach