Towards Brain-inspired System: Deep Recurrent Reinforcement Learning for Simulated Self-driving Agent

Jieneng Chen,Jingye Chen,Ruiming Zhang,Xiaobin Hu

DOI: https://doi.org/10.48550/arXiv.1903.12517

2019-03-29

Abstract:An effective way to achieve intelligence is to simulate various intelligent behaviors in the human brain. In recent years, bio-inspired learning methods have emerged, and they are different from the classical mathematical programming principle. In the perspective of brain inspiration, reinforcement learning has gained additional interest in solving decision-making tasks as increasing neuroscientific research demonstrates that significant links exist between reinforcement learning and specific neural substrates. Because of the tremendous research that focuses on human brains and reinforcement learning, scientists have investigated how robots can autonomously tackle complex tasks in the form of a self-driving agent control in a human-like way. In this study, we propose an end-to-end architecture using novel deep-Q-network architecture in conjunction with a recurrence to resolve the problem in the field of simulated self-driving. The main contribution of this study is that we trained the driving agent using a brain-inspired trial-and-error technique, which was in line with the real world situation. Besides, there are three innovations in the proposed learning network: raw screen outputs are the only information which the driving agent can rely on, a weighted layer that enhances the differences of the lengthy episode, and a modified replay mechanism that overcomes the problem of sparsity and accelerates learning. The proposed network was trained and tested under a third-partied OpenAI Gym environment. After training for several episodes, the resulting driving agent performed advanced behaviors in the given scene. We hope that in the future, the proposed brain-inspired learning system would inspire practicable self-driving control solutions.

Artificial Intelligence

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to achieve the autonomous decision - making ability of self - driving agents in the simulation environment. Specifically, the researchers proposed a brain - inspired deep recurrent reinforcement learning method to solve complex tasks in simulated self - driving control. Traditional self - driving vehicle research often relies on manually constructed data sets and supervised learning methods, which results in the data set being unable to cover all possible driving situations and being unstable when encountering unseen situations. In addition, these methods usually ignore important factors in the real world such as braking.

The main contributions of the paper are:
1. **Proposing an end - to - end architecture**: Using a new deep Q - network (DQN) architecture combined with a recurrent mechanism to deal with the simulated self - driving problem.
2. **Adopting brain - inspired trial - and - error techniques**: Training driving agents to learn how to make decisions through trial - and - error methods, which is more in line with the real - world situation.
3. **Introducing three innovation points**:
- **Only relying on the original screen output**: Abandoning intermediate game parameters, making the driving agent only rely on the original screen output.
- **Introducing a weighted layer**: Enhancing the effect of long - time sequences.
- **An improved experience replay mechanism**: Solving the problems of sparse rewards and accelerating learning.

Through these innovations, the researchers hope that the proposed brain - inspired learning system can provide inspiration for actual self - driving control solutions.