Local precision of visuotopic organization in the middle temporal area (MT) of the macaque

Thomas D. Albright,R. Desimone

DOI: https://doi.org/10.1007/BF00235981

Experimental Brain Research

Abstract:

What problem does this paper attempt to address?

Prioritized Experience-Based Reinforcement Learning With Human Guidance for Autonomous Driving

Jingda Wu,Zhiyu Huang,Wenhui Huang,Chen Lv

DOI: https://doi.org/10.1109/tnnls.2022.3177685

IF: 14.255

2022-01-01

IEEE Transactions on Neural Networks and Learning Systems

Abstract:Reinforcement learning (RL) requires skillful definition and remarkable computational efforts to solve optimization and control problems, which could impair its prospect. Introducing human guidance into RL is a promising way to improve learning performance. In this article, a comprehensive human guidance-based RL framework is established. A novel prioritized experience replay mechanism that adapts to human guidance in the RL process is proposed to boost the efficiency and performance of the RL algorithm. To relieve the heavy workload on human participants, a behavior model is established based on an incremental online learning method to mimic human actions. We design two challenging autonomous driving tasks for evaluating the proposed algorithm. Experiments are conducted to access the training and testing performance and learning mechanism of the proposed algorithm. Comparative results against the state-of-the-art methods suggest the advantages of our algorithm in terms of learning efficiency, performance, and robustness.

computer science, artificial intelligence, theory & methods,engineering, electrical & electronic, hardware & architecture
Deep Reinforcement Learning with Heuristic Corrections for UGV Navigation

Changyun Wei,Yajun Li,Yongping Ouyang,Ze Ji,Li, Yajun,Ouyang, Yongping

DOI: https://doi.org/10.1007/s10846-023-01950-y

2023-09-07

Journal of Intelligent and Robotic Systems: Theory and Applications

Abstract:Mapless navigation for mobile Unmanned Ground Vehicles (UGVs) using Deep Reinforcement Learning (DRL) has attracted significantly rising attention in robotic and related research communities. Collision avoidance from dynamic obstacles in unstructured environments, such as pedestrians and other vehicles, is one of the key challenges for mapless navigation. This paper proposes a DRL algorithm based on heuristic correction learning for autonomous navigation of a UGV in mapless configuration. We use a 24-dimensional lidar sensor, and merge the target position information and the speed information of the UGV as the input of the reinforcement learning agent. The actions of the UGV are produced as the output of the agent. Our proposed algorithm has been trained and evaluated in both static and dynamic environments. The experimental result shows that our proposed algorithm can reach the target in less time with shorter distances under the premise of ensuring safety than other algorithms. Especially, the success rate of our proposed algorithm is 2.05 times higher than the second effective algorithm and the trajectory efficiency is improved by in the dynamic environment. Finally, our proposed algorithm is deployed on a real robot in the real-world environment to validate and evaluate the algorithm performance. Experimental results show that our proposed algorithm can be directly applied to real robots robustly.
Autonomous Navigation by Mobile Robot with Sensor Fusion Based on Deep Reinforcement Learning

Yang Ou,Yiyi Cai,Youming Sun,Tuanfa Qin

DOI: https://doi.org/10.3390/s24123895

IF: 3.9

2024-06-17

Sensors

Abstract:In the domain of mobile robot navigation, conventional path-planning algorithms typically rely on predefined rules and prior map information, which exhibit significant limitations when confronting unknown, intricate environments. With the rapid evolution of artificial intelligence technology, deep reinforcement learning (DRL) algorithms have demonstrated considerable effectiveness across various application scenarios. In this investigation, we introduce a self-exploration and navigation approach based on a deep reinforcement learning framework, aimed at resolving the navigation challenges of mobile robots in unfamiliar environments. Firstly, we fuse data from the robot's onboard lidar sensors and camera and integrate odometer readings with target coordinates to establish the instantaneous state of the decision environment. Subsequently, a deep neural network processes these composite inputs to generate motion control strategies, which are then integrated into the local planning component of the robot's navigation stack. Finally, we employ an innovative heuristic function capable of synthesizing map information and global objectives to select the optimal local navigation points, thereby guiding the robot progressively toward its global target point. In practical experiments, our methodology demonstrates superior performance compared to similar navigation methods in complex, unknown environments devoid of predefined map information.

engineering, electrical & electronic,instruments & instrumentation,chemistry, analytical
Deep-Reinforcement-Learning-Based Autonomous UAV Navigation With Sparse Rewards

Xudong Zhang,Jingjing Wang,Chao Wang,Jian Wang

DOI: https://doi.org/10.1109/JIOT.2020.2973193

IF: 10.6

2020-02-11

IEEE Internet of Things Journal

Abstract:Unmanned aerial vehicles (UAVs) have the potential in delivering Internet-of-Things (IoT) services from a great height, creating an airborne domain of the IoT. In this article, we address the problem of autonomous UAV navigation in large-scale complex environments by formulating it as a Markov decision process with sparse rewards and propose an algorithm named deep reinforcement learning (RL) with nonexpert helpers (LwH). In contrast to prior RL-based methods that put huge efforts into reward shaping, we adopt the sparse reward scheme, i.e., a UAV will be rewarded if and only if it completes navigation tasks. Using the sparse reward scheme ensures that the solution is not biased toward potentially suboptimal directions. However, having no intermediate rewards hinders the agent from efficient learning since informative states are rarely encountered. To handle the challenge, we assume that a prior policy (nonexpert helper) that might be of poor performance is available to the learning agent. The prior policy plays the role of guiding the agent in exploring the state space by reshaping the behavior policy used for environmental interaction. It also assists the agent in achieving goals by setting dynamic learning objectives with increasing difficulty. To evaluate our proposed method, we construct a simulator for UAV navigation in large-scale complex environments and compare our algorithm with several baselines. Experimental results demonstrate that LwH significantly outperforms the state-of-the-art algorithms handling sparse rewards and yields impressive navigation policies comparable to those learned in the environment with dense rewards.

Engineering,Computer Science
Autonomous Navigation of UAVs in Large-Scale Complex Environments: A Deep Reinforcement Learning Approach

Chao Wang,Jian Wang,Yuan Shen,Xudong Zhang

DOI: https://doi.org/10.1109/tvt.2018.2890773

IF: 6.8

2019-01-01

IEEE Transactions on Vehicular Technology

Abstract:In this paper, we propose a deep reinforcement learning (DRL)-based method that allows unmanned aerial vehicles (UAVs) to execute navigation tasks in large-scale complex environments. This technique is important for many applications such as goods delivery and remote surveillance. The problem is formulated as a partially observable Markov decision process (POMDP) and solved by a novel online DRL algorithm designed based on two strictly proved policy gradient theorems within the actor-critic framework. In contrast to conventional simultaneous localization and mapping-based or sensing and avoidance-based approaches, our method directly maps UAVs’ raw sensory measurements into control signals for navigation. Experiment results demonstrate that our method can enable UAVs to autonomously perform navigation in a virtual large-scale complex environment and can be generalized to more complex, larger-scale, and three-dimensional environments. Besides, the proposed online DRL algorithm addressing POMDPs outperforms the state-of-the-art.
Sim-to-Real: Mapless Navigation for USVs Using Deep Reinforcement Learning

Ning Wang,Yabiao Wang,Yuming Zhao,Yong Wang,Zhigang Li

DOI: https://doi.org/10.3390/jmse10070895

IF: 2.744

2022-01-01

Journal of Marine Science and Engineering

Abstract:In recent years, mapless navigation using deep reinforcement learning algorithms has shown significant advantages in improving robot motion planning capabilities. However, the majority of past works have focused on aerial and ground robotics, with very little attention being paid to unmanned surface vehicle (USV) navigation and ultimate deployment on real platforms. In response, this paper proposes a mapless navigation method based on deep reinforcement learning for USVs. Specifically, we carefully design the observation space, action space, reward function, and neural network for a navigation policy that allows the USV to reach the destination collision-free when equipped with only local sensors. Aiming at the sim-to-real transfer and slow convergence of deep reinforcement learning, this paper proposes a dynamics-free training and consistency strategy and designs domain randomization and adaptive curriculum learning. The method was evaluated using a range of tests applied to simulated and physical environments and was proven to work effectively in a real navigation environment.
Human-in-the-Loop Deep Reinforcement Learning with Application to Autonomous Driving

Jingda Wu,Zhiyu Huang,Chao Huang,Zhongxu Hu,Peng Hang,Yang Xing,Chen Lv

2021-04-15

Abstract:Due to the limited smartness and abilities of machine intelligence, currently autonomous vehicles are still unable to handle all kinds of situations and completely replace drivers. Because humans exhibit strong robustness and adaptability in complex driving scenarios, it is of great importance to introduce humans into the training loop of artificial intelligence, leveraging human intelligence to further advance machine learning algorithms. In this study, a real-time human-guidance-based deep reinforcement learning (Hug-DRL) method is developed for policy training of autonomous driving. Leveraging a newly designed control transfer mechanism between human and automation, human is able to intervene and correct the agent's unreasonable actions in real time when necessary during the model training process. Based on this human-in-the-loop guidance mechanism, an improved actor-critic architecture with modified policy and value networks is developed. The fast convergence of the proposed Hug-DRL allows real-time human guidance actions to be fused into the agent's training loop, further improving the efficiency and performance of deep reinforcement learning. The developed method is validated by human-in-the-loop experiments with 40 subjects and compared with other state-of-the-art learning approaches. The results suggest that the proposed method can effectively enhance the training efficiency and performance of the deep reinforcement learning algorithm under human guidance, without imposing specific requirements on participant expertise and experience.

Robotics
Reinforcement Learning-Based Visual Navigation With Information-Theoretic Regularization

Qiaoyun Wu,Kai Xu,Jun Wang,Mingliang Xu,Xiaoxi Gong,Dinesh Manocha

DOI: https://doi.org/10.1109/lra.2020.3048668

IF: 5.2

2021-04-01

IEEE Robotics and Automation Letters

Abstract:To enhance the cross-target and cross-scene generalization of target-driven visual navigation based on deep reinforcement learning (RL), we introduce an information-theoretic regularization term into the RL objective. The regularization maximizes the mutual information between navigation actions and visual observation transforms of an agent, thus promoting more informed navigation decisions. This way, the agent models the action-observation dynamics by learning a variational generative model. Based on the model, the agent generates (imagines) the next observation from its current observation and navigation target. This way, the agent learns to understand the causality between navigation actions and the changes in its observations, which allows the agent to predict the next action for navigation by comparing the current and the imagined next observations. Cross-target and cross-scene evaluations on the AI2-THOR framework show that our method attains at least 10 improvement of average success rate over some state-of-the-art models. We further evaluate our model in two real-world settings: navigation in unseen indoor scenes from a discrete Active Vision Dataset (AVD) and continuous real-world environments with a TurtleBot. We demonstrate that our navigation model is able to successfully achieve navigation tasks in these scenarios.11[Online]. Available: https://github.com/wqynew/RL-based-navigation. [Online]. Available: https://github.com/wqynew/RL-based-navigation.

robotics
Navigation of Mobile Robots Based on Deep Reinforcement Learning: Reward Function Optimization and Knowledge Transfer

Weijie Li,Ming Yue,Jinyong Shangguan,Ye Jin

DOI: https://doi.org/10.1007/s12555-021-0642-7

2023-01-31

Abstract:This paper presents an end-to-end online learning navigation method based on deep reinforcement learning (DRL) for mobile robots, whose objective is that mobile robots can avoid obstacles to reach the target point in an unknown environment. Specifically, double deep Q-networks (Double DQN), dueling deep Q-networks (Dueling DQN) and prioritized experience replay (PER) are combined to form prioritized experience replay-double dueling deep Q-networks (PER-D3QN) algorithm to realize high-efficiency navigation of mobile robots. Moreover, considering the problem of sparse reward in the traditional reward function, an artificial potential field is introduced into the reward function to guide robots to fulfill the navigation task through the change of potential energy. Furthermore, in order to accelerate the training of mobile robots in complex environment, a knowledge transfer training method is proposed, which migrates the knowledge from simple to complex environment, and quickly learns on the basis of the priori knowledge. Finally, the performance is validated based on a three-dimensional simulator, which shows that the mobile robot can obtain higher rewards and achieve higher success rates and less time for navigation, indicating that the proposed approaches are feasible and efficient.

automation & control systems
Deep reinforcement learning based mapless navigation for industrial AMRs: advancements in generalization via potential risk state augmentation

Degang Xu,Peng Chen,Xianhan Zhou,Yizhi Wang,Guanzheng Tan

DOI: https://doi.org/10.1007/s10489-024-05679-5

IF: 5.3

2024-07-14

Applied Intelligence

Abstract:This article introduces a novel Deep Reinforcement Learning (DRL)-based approach for mapless navigation in Industrial Autonomous Mobile Robots, emphasizing advancements in generalization through Potential Risk State Augmentation (PRSA) and an adaptive safety optimization reward function. Traditional LiDAR-based state representations often fail to capture environmental intricacies, leading to suboptimal performance. PRSA addresses this by improving the representation of high-dimensional LiDAR data, focusing on essential risk-related information to reduce redundancy and enhance the DRL agent's generalization across various industrial settings. The adaptive reward function integrated with intrinsic reward mitigates the issue of sparse rewards in complex tasks, promoting faster learning and optimal policy convergence. Extensive experiments demonstrate that our method maintains a high success rate (over 90%) and low collision risk in narrow and dynamic environments compared to existing DRL-based methods. Meanwhile, compared with the classic navigation baseline, the proposed method improves the success rate by about 33% and reduces the mean navigation time by about 48% in real-world navigation tasks. The direct transfer of policies trained in simulations to real-world environments has demonstrated significant potential for enhancing both the efficacy and reliability of autonomous navigation.

computer science, artificial intelligence
Explainable Deep Reinforcement Learning for UAV Autonomous Navigation

Lei He,Aouf Nabil,Bifeng Song

DOI: https://doi.org/10.48550/arXiv.2009.14551

IF: 3.7

2020-09-30

Robotics

Abstract:Autonomous navigation in unknown complex environment is still a hard problem, especially for small Unmanned Aerial Vehicles (UAVs) with limited computation resources. In this paper, a neural network-based reactive controller is proposed for a quadrotor to fly autonomously in unknown outdoor environment. The navigation controller makes use of only current sensor data to generate the control signal without any optimization or configuration space searching, which reduces both memory and computation requirement. The navigation problem is modelled as a Markov Decision Process (MDP) and solved using deep reinforcement learning (DRL) method. Specifically, to get better understanding of the trained network, some model explanation methods are proposed. Based on the feature attribution, each decision making result during flight is explained using both visual and texture explanation. Moreover, some global analysis are also provided for experts to evaluate and improve the trained neural network. The simulation results illustrated the proposed method can make useful and reasonable explanation for the trained model, which is beneficial for both non-expert users and controller designer. Finally, the real world tests shown the proposed controller can navigate the quadrotor to goal position successfully and the reactive controller performs much faster than some conventional approach under the same computation resource.
Autonomous Navigation of the UAV through Deep Reinforcement Learning with Sensor Perception Enhancement

Senyan Zhao,Wei Wang,Jun Li,Subin Huang,Sanmin Liu

DOI: https://doi.org/10.1155/2023/3837615

IF: 1.43

2023-07-05

Mathematical Problems in Engineering

Abstract:The accuracy of autonomous navigation and obstacle avoidance of unmanned aerial vehicles (UAVs) in complex environments has become one challenging task. In this paper, an autonomous navigation and obstacle avoidance of the UAV (ANOAU) algorithm based on deep reinforcement learning (DRL) has been proposed to achieve accurate path planning in complex environments. In our work, we use an actor–critic-based DRL framework to achieve autonomous UAV control from sensor input to the output of the UAV's action and design a set of reward functions that can be adapted to autonomous navigation and obstacle avoidance for the UAV in the complex environment. Meanwhile, to alleviate the decision-making bias caused by the incomplete observables of the UAV, we use a gate recurrent unit network to enhance the ability to perceive the uncertain environment, enhance the perception representation and improve the accuracy of UAV real-time decision-making. Experimental simulation results verify that the ANOAU algorithm achieves good UAV flight attitude adaptive adjustment in navigation and obstacle avoidance tasks and significantly improves the generalization ability and training efficiency of the UAV navigation controller in a complex environment.

engineering, multidisciplinary,mathematics, interdisciplinary applications
Autonomous Navigation of Unmanned Vehicle Through Deep Reinforcement Learning

Letian Xu,Jiabei Liu,Haopeng Zhao,Tianyao Zheng,Tongzhou Jiang,Lipeng Liu

2024-07-18

Abstract:This paper explores the method of achieving autonomous navigation of unmanned vehicles through Deep Reinforcement Learning (DRL). The focus is on using the Deep Deterministic Policy Gradient (DDPG) algorithm to address issues in high-dimensional continuous action spaces. The paper details the model of a Ackermann robot and the structure and application of the DDPG algorithm. Experiments were conducted in a simulation environment to verify the feasibility of the improved algorithm. The results demonstrate that the DDPG algorithm outperforms traditional Deep Q-Network (DQN) and Double Deep Q-Network (DDQN) algorithms in path planning tasks.

Robotics,Machine Learning
ReinforcementDriving: Exploring Trajectories and Navigation for Autonomous Vehicles

Meng Liu,Fei Zhao,Jianwei Niu,Yu Liu

DOI: https://doi.org/10.1109/tits.2019.2960872

IF: 8.5

2021-02-01

IEEE Transactions on Intelligent Transportation Systems

Abstract:Autonomous vehicles need to solve the road keeping problem and the existing solutions based on reinforcement learning are mainly implemented in the simulators. The key of transferring the well-trained models to the real world is bridging the gaps between the simulator scenarios and the real scenarios. In this paper, we propose a method called ReinforcementDriving which explores navigation skills and trajectories from simulator for full-sized road keeping. Based on the real scenario, a driving simulator is firstly established to train an intelligent driving agent. The well-trained ReinforcementDriving agent is evaluated in a real-world scenario. We compare our work with human driving, optimal control-based tracking methods and other reinforcement learning-based lane following methods. The results demonstrate that the ReinforcementDriving system can effectively achieve lane keeping in a realistic scenario with satisfactory running time and lateral accuracy.

engineering, electrical & electronic,transportation science & technology, civil
Toward human-in-the-loop AI: Enhancing deep reinforcement learning via real-time human guidance for autonomous driving

Jingda Wu,Zhiyu Huang,Zhongxu Hu,Chen Lv

DOI: https://doi.org/10.1016/j.eng.2022.05.017

IF: 12.834

2022-07-21

Engineering

Abstract:Due to its limited intelligence and abilities, machine learning is currently unable to handle various situations thus cannot completely replace humans in real-world applications. Because humans exhibit robustness and adaptability in complex scenarios, it is crucial to introduce humans into the training loop of artificial intelligence (AI), leveraging human intelligence to further advance machine learning algorithms. In this study, a real-time human-guidance-based (Hug)-deep reinforcement learning (DRL) method is developed for policy training in an end-to-end autonomous driving case. With our newly designed mechanism for control transfer between humans and automation, humans are able to intervene and correct the agent's unreasonable actions in real time when necessary during the model training process. Based on this human-in-the-loop guidance mechanism, an improved actor-critic architecture with modified policy and value networks is developed. The fast convergence of the proposed Hug-DRL allows real-time human guidance actions to be fused into the agent's training loop, further improving the efficiency and performance of DRL. The developed method is validated by human-in-the-loop experiments with 40 subjects and compared with other state-of-the-art learning approaches. The results suggest that the proposed method can effectively enhance the training efficiency and performance of the DRL algorithm under human guidance without imposing specific requirements on participants' expertise or experience.

engineering, multidisciplinary
Improved Deep Reinforcement Learning with Expert Demonstrations for Urban Autonomous Driving

Haochen Liu,Zhiyu Huang,Jingda Wu,Chen Lv

DOI: https://doi.org/10.48550/arXiv.2102.09243

IF: 3.7

2021-02-18

Robotics

Abstract:Learning-based approaches, such as reinforcement learning (RL) and imitation learning (IL), have indicated superiority over rule-based approaches in complex urban autonomous driving environments, showing great potential to make intelligent decisions. However, current RL and IL approaches still have their own drawbacks, such as low data efficiency for RL and poor generalization capability for IL. In light of this, this paper proposes a novel learning-based method that combines deep reinforcement learning and imitation learning from expert demonstrations, which is applied to longitudinal vehicle motion control in autonomous driving scenarios. Our proposed method employs the soft actor-critic and modifies the learning process of the policy network to incorporate both the goals of maximizing reward and imitating the expert. Moreover, an adaptive prioritized experience replay is designed to sample experience from both the agent's self-exploration and expert demonstration, in order to improve sample efficiency. The proposed method is validated in a simulated urban roundabout scenario and compared with various prevailing RL and IL baselines. The results manifest that the proposed method has a faster training speed, as well as better performance in navigating safely and time-efficiently.
UAV Obstacle Avoidance by Human-in-the-Loop Reinforcement in Arbitrary 3D Environment

Xuyang Li,Jianwu Fang,Kai Du,Kuizhi Mei,Jianru Xue

DOI: https://doi.org/10.48550/arXiv.2304.05959

2023-04-07

Abstract:This paper focuses on the continuous control of the unmanned aerial vehicle (UAV) based on a deep reinforcement learning method for a large-scale 3D complex environment. The purpose is to make the UAV reach any target point from a certain starting point, and the flying height and speed are variable during navigation. In this work, we propose a deep reinforcement learning (DRL)-based method combined with human-in-the-loop, which allows the UAV to avoid obstacles automatically during flying. We design multiple reward functions based on the relevant domain knowledge to guide UAV navigation. The role of human-in-the-loop is to dynamically change the reward function of the UAV in different situations to suit the obstacle avoidance of the UAV better. We verify the success rate and average step size on urban, rural, and forest scenarios, and the experimental results show that the proposed method can reduce the training convergence time and improve the efficiency and accuracy of navigation tasks. The code is available on the website <a class="link-external link-https" href="https://github.com/Monnalo/UAV_navigation" rel="external noopener nofollow">this https URL</a>.

Robotics,Artificial Intelligence
Deep reinforcement learning-aided autonomous navigation with landmark generators

Xuanzhi Wang,Yankang Sun,Yuyang Xie,Jiang Bin,Jian Xiao

DOI: https://doi.org/10.3389/fnbot.2023.1200214

IF: 3.493

2023-08-23

Frontiers in Neurorobotics

Abstract:Mobile robots are playing an increasingly significant role in social life and industrial production, such as searching and rescuing robots, autonomous exploration of sweeping robots, and so on. Improving the accuracy of autonomous navigation of mobile robots is a hot issue to be solved. However, traditional navigation methods are unable to realize crash-free navigation in an environment with dynamic obstacles, more and more scholars are gradually using autonomous navigation based on deep reinforcement learning (DRL) to replace overly conservative traditional methods. But on the other hand, DRL's training time is too long, and the lack of long-term memory easily leads the robot to a dead end, which makes its application in the actual scene more difficult. To shorten training time and prevent mobile robots from getting stuck and spinning around, we design a new robot autonomous navigation framework which combines the traditional global planning and the local planning based on DRL. Therefore, the entire navigation process can be transformed into first using traditional navigation algorithms to find the global path, then searching for several high-value landmarks on the global path, and then using the DRL algorithm to move the mobile robot toward the designated landmarks to complete the final navigation, which makes the robot training difficulty greatly reduced. Furthermore, in order to improve the lack of long-term memory in deep reinforcement learning, we design a feature extraction network containing memory modules to preserve the long-term dependence of input features. Through comparing our methods with traditional navigation methods and reinforcement learning based on end-to-end depth navigation methods, it shows that while the number of dynamic obstacles is large and obstacles are rapidly moving, our proposed method is, on average, 20% better than the second ranked method in navigation efficiency (navigation time and navigation paths' length), 34% better than the second ranked method in safety (collision times), 26.6% higher than the second ranked method in success rate, and shows strong robustness.

robotics,computer science, artificial intelligence,neurosciences
End-to-end Autonomous Vehicle Navigation Control Method Guided by the Dynamic Window Approach

Longfei Gao,Yan Wu,Liye Wang,Lifang Wang,Junzhi Zhang,Kui Li

DOI: https://doi.org/10.1109/cieec58067.2023.10167001

2023-01-01

Abstract:Existing end-to-end vehicle navigation control methods based on deep reinforcement learning generally have low exploration efficiency and difficulty converging the model to the ideal state. To address these problems, this paper proposes a hybrid reinforcement learning framework that can fuse the traditional path planning algorithm (dynamic window approach, DWA) with the deep reinforcement learning approach. By taking advantage of DWA's ability to plan a collision-free trajectory with guaranteed vehicle dynamics constraints quickly, giving positive guidance to the DRL module at the early stage of its training, thus improving exploration efficiency while ensuring exploration breadth. To verify the effectiveness of the algorithm, a joint CARLA and ROS simulation environment is built and simulated in a typical scenario. The simulation results show that compared with existing deep reinforcement learning methods, the proposed method in this paper has significantly improved in terms of model convergence speed, stability, and pre-mid-term decision performance, in which the training time of TD3 decision network can be shortened by more than 85%.
Deep reinforcement learning navigation via decision transformer in autonomous driving

Lun Ge,Xiaoguang Zhou,Yongqiang Li,Yongcong Wang

DOI: https://doi.org/10.3389/fnbot.2024.1338189

IF: 3.493

2024-03-19

Frontiers in Neurorobotics

Abstract:In real-world scenarios, making navigation decisions for autonomous driving involves a sequential set of steps. These judgments are made based on partial observations of the environment, while the underlying model of the environment remains unknown. A prevalent method for resolving such issues is reinforcement learning, in which the agent acquires knowledge through a succession of rewards in addition to fragmentary and noisy observations. This study introduces an algorithm named deep reinforcement learning navigation via decision transformer (DRLNDT) to address the challenge of enhancing the decision-making capabilities of autonomous vehicles operating in partially observable urban environments. The DRLNDT framework is built around the Soft Actor-Critic (SAC) algorithm. DRLNDT utilizes Transformer neural networks to effectively model the temporal dependencies in observations and actions. This approach aids in mitigating judgment errors that may arise due to sensor noise or occlusion within a given state. The process of extracting latent vectors from high-quality images involves the utilization of a variational autoencoder (VAE). This technique effectively reduces the dimensionality of the state space, resulting in enhanced training efficiency. The multimodal state space consists of vector states, including velocity and position, which the vehicle's intrinsic sensors can readily obtain. Additionally, latent vectors derived from high-quality images are incorporated to facilitate the Agent's assessment of the present trajectory. Experiments demonstrate that DRLNDT may achieve a superior optimal policy without prior knowledge of the environment, detailed maps, or routing assistance, surpassing the baseline technique and other policy methods that lack historical data.

robotics,computer science, artificial intelligence,neurosciences

Local precision of visuotopic organization in the middle temporal area (MT) of the macaque

Prioritized Experience-Based Reinforcement Learning With Human Guidance for Autonomous Driving

Deep Reinforcement Learning with Heuristic Corrections for UGV Navigation

Autonomous Navigation by Mobile Robot with Sensor Fusion Based on Deep Reinforcement Learning

Deep-Reinforcement-Learning-Based Autonomous UAV Navigation With Sparse Rewards

Autonomous Navigation of UAVs in Large-Scale Complex Environments: A Deep Reinforcement Learning Approach

Sim-to-Real: Mapless Navigation for USVs Using Deep Reinforcement Learning

Human-in-the-Loop Deep Reinforcement Learning with Application to Autonomous Driving

Reinforcement Learning-Based Visual Navigation With Information-Theoretic Regularization

Navigation of Mobile Robots Based on Deep Reinforcement Learning: Reward Function Optimization and Knowledge Transfer

Deep reinforcement learning based mapless navigation for industrial AMRs: advancements in generalization via potential risk state augmentation

Explainable Deep Reinforcement Learning for UAV Autonomous Navigation

Autonomous Navigation of the UAV through Deep Reinforcement Learning with Sensor Perception Enhancement

Autonomous Navigation of Unmanned Vehicle Through Deep Reinforcement Learning

ReinforcementDriving: Exploring Trajectories and Navigation for Autonomous Vehicles

Toward human-in-the-loop AI: Enhancing deep reinforcement learning via real-time human guidance for autonomous driving

Improved Deep Reinforcement Learning with Expert Demonstrations for Urban Autonomous Driving

UAV Obstacle Avoidance by Human-in-the-Loop Reinforcement in Arbitrary 3D Environment

Deep reinforcement learning-aided autonomous navigation with landmark generators

End-to-end Autonomous Vehicle Navigation Control Method Guided by the Dynamic Window Approach

Deep reinforcement learning navigation via decision transformer in autonomous driving