DQL energy management: An online-updated algorithm and its application in fix-line hybrid electric vehicle

Runnan Zou,Likang Fan,Yanrui Dong,Siyu Zheng,Chenxing Hu

DOI: https://doi.org/10.1016/j.energy.2021.120174

IF: 9

2021-06-01

Energy

Abstract:<p>With decades' development of energy management strategy in hybrid electric vehicle, learning-based method has been deemed as a key solution for energy economy and real time. However, current energy management strategy cannot reach an optimal energy economy performance and online update in a tolerable time lag. Aiming at solving these problems, an accelerated reinforcement learning method and an online-updated strategy are proposed in present work. Firstly, prioritized replay is applied in deep Q network with normalized advantage function for a fast convergence to an optimal strategy. Prioritized replay module endows weight to trained history data which is utilized in neural network training. The neural network is updated towards optimal strategy by weight in an effective way. Secondly, the online-updated strategy for fix-line hybrid electric vehicle is designed based on the accelerated reinforcement learning method and model predictive control. The predicted future road information generated by model predictive control in each time interval is delivered to the accelerated reinforcement learning module for online energy management strategy generating. Finally, with all efforts above, the online-updated strategy is carried out and validated through hardware-in-the-loop simulation. The results show that this approach promotes the energy economic performance while updating strategy in real time.</p>

energy & fuels,thermodynamics

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to implement an online - updated energy management strategy in hybrid electric vehicles (HEVs) with fixed routes. Specifically, the current energy management strategies are insufficient in achieving optimal energy - economic performance and real - time online update, and cannot complete the optimization within an acceptable time delay. To solve these problems, this paper proposes an accelerated reinforcement learning method and an online update strategy. This method first applies the Prioritized Replay technique to the Deep Q Learning (DQL) with the Normalized Advantage Function (NAF) to quickly converge to the optimal strategy. Secondly, based on the accelerated reinforcement learning method and Model Predictive Control (MPC), an online update strategy for fixed - route hybrid electric vehicles is designed. The effectiveness of this strategy is verified by Hardware - in - the - Loop (HIL) simulation, and the results show that this method not only improves the energy - economic performance but also enables real - time strategy update. In short, this research aims to develop a strategy that can be updated in real - time and optimize the energy management of fixed - route hybrid electric vehicles by combining deep reinforcement learning and model predictive control.

DQL energy management: An online-updated algorithm and its application in fix-line hybrid electric vehicle

Online updating energy management strategy based on deep reinforcement learning with accelerated training for hybrid electric tracked vehicles

A deep reinforcement learning approach to energy management control with connected information for hybrid electric vehicles

Hierarchical reinforcement learning based energy management strategy for hybrid electric vehicle

Energy management strategy of intelligent plug-in split hybrid electric vehicle based on deep reinforcement learning with optimized path planning algorithm

Energy management for hybrid electric vehicles based on imitation reinforcement learning

A Data-Driven Reinforcement Learning Based Energy Management Strategy via Bridging Offline Initialization and Online Fine-Tuning for a Hybrid Electric Vehicle

Online mixed-integer optimal energy management strategy for connected hybrid electric vehicles

Bi-level energy management strategy for power-split plug-in hybrid electric vehicles: A reinforcement learning approach for prediction and control

Enhanced Q-learning for real-time hybrid electric vehicle energy management with deterministic rule

Heuristic action execution for energy efficient charge-sustaining control of connected hybrid vehicles with model-free double Q-learning

Multi-step Reinforcement Learning for Model-Free Predictive Energy Management of an Electrified Off-Highway Vehicle

Supervisory Control of the Hybrid Off-Highway Vehicle for Fuel Economy Improvement Using Predictive Double Q-learning with Backup Models

Rule and Q-learning Based Hybrid Energy Management for Electric Vehicle

Energy management for a hybrid electric vehicle based on prioritized deep reinforcement learning framework

Heuristic Energy Management Strategy of Hybrid Electric Vehicle Based on Deep Reinforcement Learning With Accelerated Gradient Optimization

A novel energy management strategy of hybrid electric vehicle via an improved TD3 deep reinforcement learning

Reinforcement Learning-Based Energy Management for Hybrid Electric Vehicle Enhanced by Velocity Prediction

Real-Time Energy Management for Plug-in Hybrid Electric Vehicles via Incorporating Double-Delay Q-Learning and Model Prediction Control

A reinforcement learning-based energy management strategy for fuel cell hybrid vehicle considering real-time velocity prediction

Battery life constrained real-time energy management strategy for hybrid electric vehicles based on reinforcement learning