Abstract:<p>This paper investigates a model-free supervisory control methodology with double Q-learning for the hybrid vehicle in charge-sustaining scenarios. It aims to improve the vehicle's energy efficiency continuously while maintaining the battery's state-of-charge in real-world driving. Two new heuristic action execution policies, the max-value-based policy and the random policy, are proposed for the double Q-learning method to reduce overestimation of the merit-function values for each action in power-split control of the vehicle. Experimental studies based on software-in-the-loop (offline learning) and hardware-in-the-loop (online learning) platforms are carried out to explore the potential of energy-saving in four driving cycles defined with real-world vehicle operations. The results from 35 rounds of offline undisturbed learning show that the heuristic action execution policies can improve the learning performance of conventional double Q-learning by achieving at least 1.09% higher energy efficiency. The proposed methods achieve similar results obtained by dynamic programming, but they have the capability of real-time online application. Double Q-learnings are shown more robust to turbulence during the disturbed learning: they realise at least three times improvement in energy efficiency compared to the standard Q-learning. Random execution policy achieves 1.18% higher energy efficiency than the max-value-based policy for the same driving condition. Significant tests show that deciding factor in the random execution policy has little impact on learning performance. By implementing the control strategies for online learning, the proposed model-free control method can save energy by more than 4.55% in the predefined real-world driving conditions compared to the method using standard Q-learning.</p>

Double-Q Learning-Based Dvfs For Multi-Core Real-Time Systems

A DVFS Design and Simulation Framework Using Machine Learning Models

Multi-core Chip Dynamic Power Management Framework Based on Reinforcement Learning br

User- and Process-Driven Dynamic Voltage and Frequency Scaling.

An Efficient and Flexible Learning Framework for Dynamic Power and Thermal Co-Management

Power Reduction Through Measurement and Modeling of Users and CPUs: Summary

An Enhanced Dynamic Voltage Scaling Scheme for Energy-Efficient Embedded Real-Time Control Systems

Learning-Directed Dynamic Voltage and Frequency Scaling Scheme with Adjustable Performance for Single-Core and Multi-Core Embedded and Mobile Systems

Coordinated management of DVFS and cache partitioning under QoS constraints to save energy in multi-core systems

Performance-Aware Power Management in Embedded Controllers with Multiple-Voltage Processors

Feedback Scheduling of Real-Time Control Tasks in Power-Aware Embedded Systems

A multi-agent reinforcement learning-based method for server energy efficiency optimization combining DVFS and dynamic fan control

Extending Performance-Energy Trade-offs Via Dynamic Core Scaling

Real-Time Dynamic Voltage Loop Scheduling for Multi-Core Embedded Systems

Load Importance-Based DVFS Scheme for Interactive Systems

Modular reinforcement learning for self-adaptive energy efficiency optimization in multicore system

Energy Consumption Optimization DVFS Mechanism for Commercial Processors

Enhanced Energy-Aware Feedback Scheduling of Embedded Control Systems

Heuristic action execution for energy efficient charge-sustaining control of connected hybrid vehicles with model-free double Q-learning

DSO: A GPU Energy Efficiency Optimizer by Fusing Dynamic and Static Information

Energy-Efficient Computation with DVFS using Deep Reinforcement Learning for Multi-Task Systems in Edge Computing