Abstract:To solve air pollution problems and reduce energy costs, hybrid electric vehicles (HEV) are applied for larger scale as a superior solution and their energy management strategies (EMS) are essential for achieving higher fuel efficiency. This paper proposes a reinforcement learning (RL) based real-time EMS for a HEV, which requires no prior knowledge of the future driving conditions. First, considering that the practical driving conditions are mutative, an online recursive Markov Chain model is constructed, which can depict the stochastic environment more accurately. Then, a novel RL framework, heuristic search, is introduced, which means using extra knowledge in the search process. It aims to find an optimal action for a specific state rather than learn a policy for an entire driving cycle like traditional RL algorithms. Consequently, the learning process is significantly efficient and real-time optimization can be realized. Moreover, a Q-table is learned offline, and is used online as a heuristic function to direct the search process, which can accelerate the convergence rate and improve the real-time performance further. In the simulation, the effectiveness of the recursive Markov Chain model and the superiority of the heuristic function are verified. Then, a comparative simulation of heuristic search, Q-learning, dynamic programming (DP) and equivalent consumption minimization strategy is conducted and evaluated. The proposed method consumes 1.63% more fuel compared with DP in a totally unknown cycle, which demonstrates its control performance and real-time capability.

Energy Management for a Hybrid Electric Vehicle Based on Blended Reinforcement Learning with Backward Focusing and Prioritized Sweeping

Real-Time Energy Management for a Hybrid Electric Vehicle Based on Heuristic Search

Battery life constrained real-time energy management strategy for hybrid electric vehicles based on reinforcement learning

Online-Learning Adaptive Energy Management for Hybrid Electric Vehicles in Various Driving Scenarios Based on Dyna Framework

An adaptive hierarchical energy management strategy for hybrid electric vehicles combining heuristic domain knowledge and data-driven deep reinforcement learning

Deep Reinforcement Learning-Based Energy Management for a Series Hybrid Electric Vehicle Enabled by History Cumulative Trip Information

Heuristic Energy Management Strategy of Hybrid Electric Vehicle Based on Deep Reinforcement Learning With Accelerated Gradient Optimization

A novel energy management strategy of hybrid electric vehicle via an improved TD3 deep reinforcement learning

Deep Reinforcement Learning-Based Energy Management Strategy for Hybrid Electric Vehicles

Rule-interposing Deep Reinforcement Learning Based Energy Management Strategy for Power-Split Hybrid Electric Vehicle

Double Deep Reinforcement Learning-Based Energy Management for a Parallel Hybrid Electric Vehicle with Engine Start–Stop Strategy

Reinforcement Learning-Based Real-Time Intelligent Energy Management for Hybrid Electric Vehicles in a Model Predictive Control Framework

Human-like Energy Management Based on Deep Reinforcement Learning and Historical Driving Experiences

A Comparative Study of Deep Reinforcement Learning Based Energy Management Strategy for Hybrid Electric Vehicle

Energy Management of Hybrid Electric Vehicles Based on Model Predictive Control and Deep Reinforcement Learning

Deep Reinforcement Learning-Based Energy Management for Hybrid Electric Vehicles

Deep Reinforcement Learning Based Energy Management of Hybrid Electric Vehicles with Expert Knowledge

Adaptive real-time energy management strategy using heuristic search for off-road hybrid electric vehicles

Hierarchical reinforcement learning based energy management strategy for hybrid electric vehicle

Reinforcement learning-based energy management strategies of fuel cell hybrid vehicles with multi-objective control

Energy Management Strategy Based on Deep Reinforcement Learning and Speed Prediction for Power-Split Hybrid Electric Vehicle with Multidimensional Continuous Control