Abstract:<p>This paper investigates a model-free supervisory control methodology with double Q-learning for the hybrid vehicle in charge-sustaining scenarios. It aims to improve the vehicle's energy efficiency continuously while maintaining the battery's state-of-charge in real-world driving. Two new heuristic action execution policies, the max-value-based policy and the random policy, are proposed for the double Q-learning method to reduce overestimation of the merit-function values for each action in power-split control of the vehicle. Experimental studies based on software-in-the-loop (offline learning) and hardware-in-the-loop (online learning) platforms are carried out to explore the potential of energy-saving in four driving cycles defined with real-world vehicle operations. The results from 35 rounds of offline undisturbed learning show that the heuristic action execution policies can improve the learning performance of conventional double Q-learning by achieving at least 1.09% higher energy efficiency. The proposed methods achieve similar results obtained by dynamic programming, but they have the capability of real-time online application. Double Q-learnings are shown more robust to turbulence during the disturbed learning: they realise at least three times improvement in energy efficiency compared to the standard Q-learning. Random execution policy achieves 1.18% higher energy efficiency than the max-value-based policy for the same driving condition. Significant tests show that deciding factor in the random execution policy has little impact on learning performance. By implementing the control strategies for online learning, the proposed model-free control method can save energy by more than 4.55% in the predefined real-world driving conditions compared to the method using standard Q-learning.</p>

Supervisory Control of the Hybrid Off-Highway Vehicle for Fuel Economy Improvement Using Predictive Double Q-learning with Backup Models

Heuristic action execution for energy efficient charge-sustaining control of connected hybrid vehicles with model-free double Q-learning

Bi-level energy management strategy for power-split plug-in hybrid electric vehicles: A reinforcement learning approach for prediction and control

Learning-based supervisory control of dual mode engine-based hybrid electric vehicle with reliance on multivariate trip information

Co-optimization strategy of unmanned hybrid electric tracked vehicle combining eco-driving and simultaneous energy management

DQL energy management: An online-updated algorithm and its application in fix-line hybrid electric vehicle

Real-Time Energy Management for Plug-in Hybrid Electric Vehicles via Incorporating Double-Delay Q-Learning and Model Prediction Control

A deep reinforcement learning approach to energy management control with connected information for hybrid electric vehicles

Energy management strategy for power-split plug-in hybrid electric vehicle based on MPC and double Q-learning

Self-supervised reinforcement learning-based energy management for a hybrid electric vehicle

Optimal Energy Management of Plug-in Hybrid Vehicles Through Exploration-to-Exploitation Ratio Control in Ensemble Reinforcement Learning

Enhanced Q-learning for real-time hybrid electric vehicle energy management with deterministic rule

An Energy Management Strategy of Power-Split Hybrid Electric Vehicles Using Reinforcement Learning

Multi-step Reinforcement Learning for Model-Free Predictive Energy Management of an Electrified Off-Highway Vehicle

Dual-loop Online Intelligent Programming for Driver-Oriented Predict Energy Management of Plug-in Hybrid Electric Vehicles

Optimizing Hybrid Electric Vehicle Coupling Organic Rankine Cycle Energy Management Strategy Via Deep Reinforcement Learning

Deep Reinforcement Learning based Energy Management for Heavy Duty HEV considering Discrete-Continuous Hybrid Action Space

Hierarchical reinforcement learning based energy management strategy for hybrid electric vehicle

A novel power management strategy for hybrid off-road vehicles

Online Learning Control for Hybrid Electric Vehicle

Online updating energy management strategy based on deep reinforcement learning with accelerated training for hybrid electric tracked vehicles