Abstract:<p>Power allocation is of crucial significance to energy management system in the hybrid construction machinery (HCM). Most of the existing HCM energy management strategies are only formulated based on the predefined rules, which causes the system unable to adapt to the changeable and complicated working conditions, thus seriously limiting the energy saving potential of hybrid technology. In this paper, we build a reinforcement learning-based intelligent energy management architecture for HCM. Given the working conditions and operating characteristics of HCM, a Q-function updating method combining direct learning and indirect learning is proposed to enhance the performance and practicability of reinforcement learning. A virtual world model (VWM) is introduced to approximate the real-world environment and facilitate the identification of data-driven environment, so as to enhance the real-time performance and adaptability of the architecture. Based on the characteristics of HCM working conditions, the load cycle is subdivided, and the stationary Markov chain is employed to yield real-time transfer probability matrices of required power to accelerate the updating of the environment model. An HCM experiment platform is built, in which the typical signal of working condition is sampled for simulation. The results indicate that DYNA-Q based architecture outperforms Q-learning and rule-based strategy (RBS) in terms of adaptivity, real-time performance and optimality. The results also demonstrate that with the proposed architecture, the working condition of internal combustion engine (ICE) and the charge-discharge of ultracapacitor are more rational and efficient.</p>

Real-time Energy Management Controller Design for a Hybrid Excavator Using Reinforcement Learning

Control Strategy of Hybrid Excavator Based on Condition Recognition and Hybridization Optimization

Research on the Design and Control Strategy for a Flow-Coupling-based Hydraulic Hybrid Excavator

A Novel Hybrid Control Strategy for Potential Energy Regeneration Systems of Hybrid Hydraulic Excavators.

Fuzzy Logic Control for a Hydraulic Hybrid Excavator Based on Torque Prediction and Genetic Algorithm Optimization

Prediction-based Stochastic Dynamic Programming Control for Excavator

An adaptive hierarchical energy management strategy for hybrid electric vehicles combining heuristic domain knowledge and data-driven deep reinforcement learning

A novel power management strategy for hybrid off-road vehicles

Battery life constrained real-time energy management strategy for hybrid electric vehicles based on reinforcement learning

Reinforcement learning-based intelligent energy management architecture for hybrid construction machinery

Safe Deep Reinforcement Learning-Based Constrained Optimal Control Scheme for HEV Energy Management

Reinforcement learning-based real-time power management for hybrid energy storage system in the plug-in hybrid electric vehicle

A deep reinforcement learning approach to energy management control with connected information for hybrid electric vehicles

Hierarchical reinforcement learning based energy management strategy for hybrid electric vehicle

Mixed-Integer Optimal Control via Reinforcement Learning: A Case Study on Hybrid Electric Vehicle Energy Management

A novel energy management strategy of hybrid electric vehicle via an improved TD3 deep reinforcement learning

Adaptive Energy Management for Real Driving Conditions via Transfer Reinforcement Learning

Data-driven modeling and supervisory control system optimization for plug-in hybrid electric vehicles

Mixed‐Integer Optimal Control via Reinforcement Learning: A Case Study on Hybrid Electric Vehicle Energy Management

Adaptive real-time energy management strategy using heuristic search for off-road hybrid electric vehicles

Reinforcement Learning for Optimal Control of a District Cooling Energy Plant