Abstract:Our research investigates the challenges Deep Reinforcement Learning (DRL) faces in complex, Partially Observable Markov Decision Processes (POMDP) such as autonomous driving (AD), and proposes a solution for vision-based navigation in these environments. Partial observability reduces RL performance significantly, and this can be mitigated by augmenting sensor information and data fusion to reflect a more Markovian environment. However, this necessitates an increasingly complex perception module, whose training via RL is complicated due to inherent limitations. As the neural network architecture becomes more complex, the reward function's effectiveness as an error signal diminishes since the only source of supervision is the reward, which is often noisy, sparse, and delayed. Task-irrelevant elements in images, such as the sky or certain objects, pose additional complexities. Our research adopts an offline-trained encoder to leverage large video datasets through self-supervised learning to learn generalizable representations. Then, we train a head network on top of these representations through DRL to learn to control an ego vehicle in the CARLA AD simulator. This study presents a broad investigation of the impact of different learning schemes for offline-training of encoders on the performance of DRL agents in challenging AD tasks. Furthermore, we show that the features learned by watching BDD100K driving videos can be directly transferred to achieve lane following and collision avoidance in CARLA simulator, in a zero-shot learning fashion. Finally, we explore the impact of various architectural decisions for the RL networks to utilize the transferred representations efficiently. Therefore, in this work, we introduce and validate an optimal way for obtaining suitable representations of the environment, and transferring them to RL networks.

Deep Reinforcement Learning Using a Low-Dimensional Observation Filter for Visual Complex Video Game Playing

Deep Reinforcement Learning for Autonomous Driving by Transferring Visual Features.

A Survey of Deep Reinforcement Learning in Video Games

Diagnosing and exploiting the computational demands of videos games for deep reinforcement learning

Playing a FPS Doom Video Game with Deep Visual Reinforcement Learning

Image Quality Assessment in Visual Reinforcement Learning for Fast-moving Targets

Deep Reinforcement Learning: A Brief Survey

Differentially Encoded Observation Spaces for Perceptive Reinforcement Learning

Depth-CUPRL: Depth-Imaged Contrastive Unsupervised Prioritized Representations in Reinforcement Learning for Mapless Navigation of Unmanned Aerial Vehicles

Enhanced Low-Dimensional Sensing Mapless Navigation of Terrestrial Mobile Robots Using Double Deep Reinforcement Learning Techniques

An Examination of Offline-Trained Encoders in Vision-Based Deep Reinforcement Learning for Autonomous Driving

Double Deep Reinforcement Learning Techniques for Low Dimensional Sensing Mapless Navigation of Terrestrial Mobile Robots

Image-Based Deep Reinforcement Learning with Intrinsically Motivated Stimuli: On the Execution of Complex Robotic Tasks

Become a Proficient Player with Limited Data through Watching Pure Videos

Learning Sparse Control Tasks from Pixels by Latent Nearest-Neighbor-Guided Explorations

Visual Diagnostics for Deep Reinforcement Learning Policy Development

Visual Kinematics Representation Via Self-Supervised Object Tracking for Deep Reinforcement Learning

Deep Learning for Video Game Playing

Interactive Learning with Corrective Feedback for Policies based on Deep Neural Networks

Robot Control in Human Environment Using Deep Reinforcement Learning and Convolutional Neural Network.

Transformer Based Reinforcement Learning For Games