Abstract:Chiplet technology has emerged as a promising solution to address the increasing demand for high-performance computing in light of the slowdown of Moore's law. While chiplet-based multicore systems offer higher performance through heterogeneous integration, they also pose challenges for power delivery system (PDS) design. The integration of additional vertical and inter-chiplet connections, along with higher power density, impose stringent requirements on power delivery. Moreover, PDS efficiency is affected by workload variations at runtime, necessitating the need to design and manage PDSs and processors as a whole to improve system energy efficiency while balancing performance. In this article, we propose an offline-online co-design optimization methodology that combines offline PDS design optimization with online power management. To address the power consumption and delivery mismatch, we introduce a centralized deep Q-network (DQN)-based online control scheme for power co-management in chiplet-based multicore systems. By carefully designing the state space and reward functions, our approach achieves workload-aware adaptive control to reduce the energy-delay-product (EDP) while maintaining PDS efficiency under a given performance target (PT). We conduct evaluations on realistic applications to validate the effectiveness of our approach. For 64-core systems, our method achieves an average EDP reduction of 67% while meeting a 90% PT, surpassing state-of-the-art modular Q-learning (MQL)-based and heuristic-based approaches by up to 4% and 16%, respectively. Additionally, our approach demonstrates wiser action selection policies, higher control stability, and lower implementation overhead compared to the MQL-based approach.

Improve the Stability and Robustness of Power Management through Model-free Deep Reinforcement Learning

An Efficient and Flexible Learning Framework for Dynamic Power and Thermal Co-Management

Multi-core Chip Dynamic Power Management Framework Based on Reinforcement Learning br

Power Management for Chiplet-Based Multicore Systems Using Deep Reinforcement Learning

Multi-Core Power Management through Deep Reinforcement Learning

Deep Reinforcement Learning-Based Power Management for Chiplet-Based Multicore Systems

Autonomous Power Management With Double- Q Reinforcement Learning Method

Smart Knowledge Transfer-based Runtime Power Management

Online Power Management for Multi-Cores: A Reinforcement Learning Based Approach

An Integrated Demand Response-Based Energy Management Strategy for Integrated Energy System Based on Deep Reinforcement Learning

Q-DPM: an Efficient Model-Free Dynamic Power Management Technique

Optimizing Data Center Energy Efficiency Via Event-Driven Deep Reinforcement Learning

Model-Free Real-Time Autonomous Energy Management for a Residential Multi-Carrier Energy System: A Deep Reinforcement Learning Approach

Reinforcement learning-based real-time power management for hybrid energy storage system in the plug-in hybrid electric vehicle

Online Learning Based on a Novel Cost Function for System Power Management

A Holistic Power Optimization Approach for Microgrid Control Based on Deep Reinforcement Learning

Modular reinforcement learning for self-adaptive energy efficiency optimization in multicore system

Deep Reinforcement Learning for Energy-Efficient on the Heterogeneous Computing Architecture

Real-Time Energy Management in Smart Homes Through Deep Reinforcement Learning

Optimizing Energy Efficiency for Data Center via Parameterized Deep Reinforcement Learning

Physical model-assisted deep reinforcement learning for energy management optimization of industrial electric-hydrogen coupling system with hybrid energy storage