Abstract:Our research investigates the challenges Deep Reinforcement Learning (DRL) faces in complex, Partially Observable Markov Decision Processes (POMDP) such as autonomous driving (AD), and proposes a solution for vision-based navigation in these environments. Partial observability reduces RL performance significantly, and this can be mitigated by augmenting sensor information and data fusion to reflect a more Markovian environment. However, this necessitates an increasingly complex perception module, whose training via RL is complicated due to inherent limitations. As the neural network architecture becomes more complex, the reward function's effectiveness as an error signal diminishes since the only source of supervision is the reward, which is often noisy, sparse, and delayed. Task-irrelevant elements in images, such as the sky or certain objects, pose additional complexities. Our research adopts an offline-trained encoder to leverage large video datasets through self-supervised learning to learn generalizable representations. Then, we train a head network on top of these representations through DRL to learn to control an ego vehicle in the CARLA AD simulator. This study presents a broad investigation of the impact of different learning schemes for offline-training of encoders on the performance of DRL agents in challenging AD tasks. Furthermore, we show that the features learned by watching BDD100K driving videos can be directly transferred to achieve lane following and collision avoidance in CARLA simulator, in a zero-shot learning fashion. Finally, we explore the impact of various architectural decisions for the RL networks to utilize the transferred representations efficiently. Therefore, in this work, we introduce and validate an optimal way for obtaining suitable representations of the environment, and transferring them to RL networks.

Image Quality Assessment in Visual Reinforcement Learning for Fast-moving Targets

Deep Reinforcement Learning-Based DQN Agent Algorithm for Visual Object Tracking in a Virtual Environmental Simulation

Efficient Camera Exposure Control for Visual Odometry via Deep Reinforcement Learning

Reinforcement Learning Meets Visual Odometry

Image-Based Deep Reinforcement Learning with Intrinsically Motivated Stimuli: On the Execution of Complex Robotic Tasks

Visual Diagnostics for Deep Reinforcement Learning Policy Development

Deep Reinforcement Learning Using a Low-Dimensional Observation Filter for Visual Complex Video Game Playing

Attention-Based Deep Reinforcement Learning for Virtual Cinematography of 360 Videos

Visual Reinforcement Learning with Self-Supervised 3D Representations

Visual Kinematics Representation Via Self-Supervised Object Tracking for Deep Reinforcement Learning

Learning Quadrotor Control From Visual Features Using Differentiable Simulation

An Examination of Offline-Trained Encoders in Vision-Based Deep Reinforcement Learning for Autonomous Driving

Real-time visual tracking by deep reinforced decision making

When Should We Prefer State-to-Visual DAgger Over Visual Reinforcement Learning?

Stabilizing Visual Reinforcement Learning Via Asymmetric Interactive Cooperation

Masked and Inverse Dynamics Modeling for Data-Efficient Reinforcement Learning

Deep introspective SLAM: deep reinforcement learning based approach to avoid tracking failure in visual SLAM

Low-quality Image Object Detection Based on Reinforcement Learning Adaptive Enhancement

Learning to Prevent Monocular SLAM Failure using Reinforcement Learning

Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control

Multi-Modal Deep Reinforcement Learning for Edge-Assisted Video Analytics.