Abstract:Reinforcement learning is a promising approach to solve the energy management for hybrid electric vehicles. In this paper, based on the DQN (Deep Q-Network) reinforcement learning algorithm which is widely used at present, double DQN, dueling DQN and learning from demonstration are integrated; states, actions, rewards and the experience pool based on the characteristics of series-parallel multi-speed hybrid powertrain are designed; the hybrid energy management strategy based on D4QN (Double Dueling Deep Q-Network with Demonstrations) algorithm is established. Based on the training results of D4QN algorithm, multi-parameter analysis under state and action space, HCU (Hybrid control unit) application and MIL (Model in-loop) test research are conducted. The results show that the D4QN algorithm can achieve both the approximate global optimal results, which differs from the result of dynamic programming by 0.05% under the training cycle, and establish the positive mapping relationship between state variables and action variables of the hybrid powertrain with excellent generalization ability, which can be applied to HCU and control the hybrid electric vehicle effectively in real time. Compared with the original rule-based energy management strategy for the hybrid electric vehicle, the fuel economy under WLTC cycle is improved by 9.64% after the application of D4QN energy management strategy. The proposed strategy can realize a termination of SOC (State of charge) within 50±10% under different initial SOC states for a battery of 1.8kWh, achieving the goal of SOC robustness. In addition, the proposed strategy has strong adaptability to the unknown cycles; the fuel economy of the vehicle after the application of D4QN energy management strategy in the untrained NEDC and CLTC-P cycles is improved by more than 10% compared to the WLTC cycle.

Reinforcement Learning-Based Energy Management Control Strategy of Hybrid Electric Vehicles

A deep reinforcement learning approach to energy management control with connected information for hybrid electric vehicles

Heuristic Energy Management Strategy of Hybrid Electric Vehicle Based on Deep Reinforcement Learning With Accelerated Gradient Optimization

Hierarchical reinforcement learning based energy management strategy for hybrid electric vehicle

Deep Reinforcement Learning Based Energy Management for a Hybrid Electric Vehicle

Research on Deep Reinforcement Learning-based Intelligent Car-following Control and Energy Management Strategy for Hybrid Electric Vehicles

Continuous Reinforcement Learning of Energy Management with Deep Q Network for a Power Split Hybrid Electric Bus

Deep Reinforcement Learning-Based Energy Management Strategy for Hybrid Electric Vehicles

Energy Management Based on D4QN Reinforcement Learning for a Series-Parallel Multi-Speed Hybrid Electric Vehicle

Deep Reinforcement Learning Based Adaptive Energy Management for Plug-in Hybrid Electric Vehicle with Double Deep Q-network

Continuous Reinforcement Learning-Based Energy Management Strategy for Hybrid Electric-Tracked Vehicles

Reinforcement Learning Energy Management for Hybrid Electric Tracked Vehicle with Deep Deterministic Policy Gradient

Energy Management of Hybrid Electric Vehicles Based on Model Predictive Control and Deep Reinforcement Learning

Multi-Agent Deep Reinforcement Learning-Based Multi-Objective Cooperative Control Strategy for Hybrid Electric Vehicles

Multi-objective Control and Energy Management Strategy Based on Deep Q-network for Parallel Hybrid Electric Vehicles

Reinforcement Learning-Based Heuristic Planning for Optimized Energy Management in Power-Split Hybrid Electric Heavy Duty Vehicles

Fast Learning-based Control for Energy Management of Hybrid Electric Vehicles

Reinforcement Learning–Based Energy Management Strategy for a Hybrid Electric Tracked Vehicle

Energy Management Strategy Based on Deep Reinforcement Learning and Speed Prediction for Power-Split Hybrid Electric Vehicle with Multidimensional Continuous Control

An Energy Management Strategy of Power-Split Hybrid Electric Vehicles Using Reinforcement Learning

Energy Management Strategy Based on Deep Reinforcement Learning and Speed Prediction for Power‐split Hybrid Electric Vehicle with Multi‐dimensional Continuous Control