Abstract:In real-world scenarios, making navigation decisions for autonomous driving involves a sequential set of steps. These judgments are made based on partial observations of the environment, while the underlying model of the environment remains unknown. A prevalent method for resolving such issues is reinforcement learning, in which the agent acquires knowledge through a succession of rewards in addition to fragmentary and noisy observations. This study introduces an algorithm named deep reinforcement learning navigation via decision transformer (DRLNDT) to address the challenge of enhancing the decision-making capabilities of autonomous vehicles operating in partially observable urban environments. The DRLNDT framework is built around the Soft Actor-Critic (SAC) algorithm. DRLNDT utilizes Transformer neural networks to effectively model the temporal dependencies in observations and actions. This approach aids in mitigating judgment errors that may arise due to sensor noise or occlusion within a given state. The process of extracting latent vectors from high-quality images involves the utilization of a variational autoencoder (VAE). This technique effectively reduces the dimensionality of the state space, resulting in enhanced training efficiency. The multimodal state space consists of vector states, including velocity and position, which the vehicle's intrinsic sensors can readily obtain. Additionally, latent vectors derived from high-quality images are incorporated to facilitate the Agent's assessment of the present trajectory. Experiments demonstrate that DRLNDT may achieve a superior optimal policy without prior knowledge of the environment, detailed maps, or routing assistance, surpassing the baseline technique and other policy methods that lack historical data.

Rethinking Decision Transformer via Hierarchical Reinforcement Learning

Decision Transformer: Reinforcement Learning via Sequence Modeling

Deep reinforcement learning navigation via decision transformer in autonomous driving

In-Context Decision Transformer: Reinforcement Learning via Hierarchical Chain-of-Thought

On Transforming Reinforcement Learning by Transformer: The Development Trajectory

Decision Mamba: Reinforcement Learning via Hybrid Selective Sequence Modeling

Graph Decision Transformer

Predictive Coding for Decision Transformer

On Transforming Reinforcement Learning With Transformers: The Development Trajectory

How Crucial is Transformer in Decision Transformer?

DODT: Enhanced Online Decision Transformer Learning through Dreamer's Actor-Critic Trajectory Forecasting

Hierarchical Prompt Decision Transformer: Improving Few-Shot Policy Generalization with Global and Adaptive Guidance

Mobile Robot Sequential Decision Making Using a Deep Reinforcement Learning Hyper-Heuristic Approach

Critic-Guided Decision Transformer for Offline Reinforcement Learning

Large Sequence Models for Sequential Decision-Making: A Survey

Decision Transformer as a Foundation Model for Partially Observable Continuous Control

Decision ConvFormer: Local Filtering in MetaFormer is Sufficient for Decision Making

Rethinking Transformers in Solving POMDPs

Mamba as Decision Maker: Exploring Multi-scale Sequence Modeling in Offline Reinforcement Learning

Decision Making for Autonomous Driving Via Multimodal Transformer and Deep Reinforcement Learning * .

Solving time-delay issues in reinforcement learning via transformers