Abstract:In the residential multicarrier energy system (RMES), autonomous energy management (AEM) is beneficial for prosumers' costs that actively controls generation, energy conversion, and storage in real-time. Conventional model-based AEM methods rely on forecast models of distributed energy resources and proper system parameters, which are difficult for practical application. This article proposes a novel model-free two-timescale real-time AEM strategy for the RMES. The energy management problem in the RMES is time-decomposed into two timescales (hourly-ahead external energy trading and 15-min ahead internal energy conversion), formulated as Markov Games. Then, defining a multi-agent system not only enables separate energy management diagrams but also accelerates learning. Intelligent agents account for optimizing energy trading and energy conversion to minimize daily costs under the training of a deep deterministic policy gradient algorithm. In order to learn a collaborative strategy, all agents are trained centrally while being executed based on local information in a decentralized manner for fast response. A deterministic study indicates that the proposed AEM strategy can effectively and flexibly schedule the operation of components with respect to different price signals and load profiles. In the stochastic study that considers the error thresholds of solar panels generation and loads, the difference between the energy cost of test scenarios with a trained strategy and the no-regret learning method is equal to 0.32%, which is appropriate. In addition, the proposed method not only achieves a reduction in power imbalance but also exhibits lower energy costs when compared to various benchmark methods.

A Coordinated Energy Management Method for 5G Base Station Using Multi-Agent Deep Deterministic Policy Gradient*

Coordinated Optimization for Energy Efficient Thermal Management of 5G Base Station Site

Hierarchical Energy Management of DC Microgrid with Photovoltaic Power Generation and Energy Storage for 5G Base Station

Hybrid Control Strategy for 5G Base Station Virtual Battery-Assisted Power Grid Peak Shaving

Cost Efficiency Optimization for Multi-Cell Systems Powered by Micro-Grids

5G Ultradense Cellular-Network-Based Edge Demand Response: Energy Consumption Reduction

Coordination of Macro Base Stations for 5G Network with User Clustering

Dynamic Energy Management for Smart-Grid-Powered Coordinated Multipoint Systems

GCD Optimization and Intelligent Management for Green Base Station with Battery Control

Cost-Efficient Cellular Networks Powered by Micro-Grids

Research on decentralized resource operation optimization of virtual power plant with 5G base station

Multi-agent Reinforcement Learning for Energy Saving in Multi-Cell Massive MIMO Systems

Spatial-Temporal Energy Management of Base Stations in Cellular Networks

Optimal operation of regional integrated energy system based on multi-agent deep deterministic policy gradient algorithm

BESS Aided Reconfigurable Energy Supply using Deep Reinforcement Learning for 5G and Beyond

Adaptive Dynamic Programming for Energy-Efficient Base Station Cell Switching

Coordinated scheduling of 5G base station energy storage for voltage regulation in distribution networks

A collaborative optimization strategy for computing offloading and resource allocation based on multi-agent deep reinforcement learning

Large-scale deep reinforcement learning method for energy management of power supply units considering regulation mileage payment

Two-timescale autonomous energy management strategy based on multi-agent deep reinforcement learning approach for residential multicarrier energy system

Energy consumption optimization of 5G base stations considering variable threshold sleep mechanism