Abstract:Smart energy networks provide an effective means to accommodate high penetrations of variable renewable energy sources like solar and wind, which are key for the deep decarbonisation of energy production. However, given the variability of the renewables as well as the energy demand, it is imperative to develop effective control and energy storage schemes to manage the variable energy generation and achieve desired system economics and environmental goals. In this paper, we introduce a hybrid energy storage system composed of battery and hydrogen energy storage to handle the uncertainties related to electricity prices, renewable energy production, and consumption. We aim to improve renewable energy utilisation and minimise energy costs and carbon emissions while ensuring energy reliability and stability within the network. To achieve this, we propose a multi-agent deep deterministic policy gradient approach, which is a deep reinforcement learning-based control strategy to optimise the scheduling of the hybrid energy storage system and energy demand in real time. The proposed approach is model-free and does not require explicit knowledge and rigorous mathematical models of the smart energy network environment. Simulation results based on real-world data show that (i) integration and optimised operation of the hybrid energy storage system and energy demand reduce carbon emissions by 78.69%, improve cost savings by 23.5%, and improve renewable energy utilisation by over 13.2% compared to other baseline models; and (ii) the proposed algorithm outperforms the state-of-the-art self-learning algorithms like the deep-Q network.

Intelligent Energy Management for an On-Grid Hydrogen Refueling Station Based on Dueling Double Deep Q Network Algorithm with NoisyNet

Research on energy management strategy of heavy-duty fuel cell hybrid vehicles based on dueling-double-deep Q-network

Research on Energy Management in Hydrogen–Electric Coupled Microgrids Based on Deep Reinforcement Learning

Deep Reinforcement Learning-Based Charging Price Determination Considering the Coordinated Operation of Hydrogen Fuel Cell Electric Vehicle, Power Network and Transportation Network

Real-time optimization of large-scale hydrogen production systems using off-grid renewable energy: Scheduling strategy based on deep reinforcement learning

Federated Deep Q-Network for Multiple Microgrids Energy Management

Optimization strategy based on deep reinforcement learning for home energy management

Rough Knowledge Enhanced Dueling Deep Q-network for Household Integrated Demand Response Optimization

Approximate Cost-Optimal Energy Management of Hydrogen Electric Multiple Unit Trains Using Double Q-Learning Algorithm

Intelligent hydrogen-ammonia combined energy storage system with deep reinforcement learning

Hierarchical reinforcement learning based energy management strategy for hybrid electric vehicle

Intelligent Multi-Microgrid Energy Management Based on Deep Neural Network and Model-Free Reinforcement Learning

Intelligent Microgrid Energy Management Method Based on Deep Q Network (DQN)

A deep reinforcement learning approach to energy management control with connected information for hybrid electric vehicles

Double Deep Q-learning Based Real-Time Optimization Strategy for Microgrids

Electric vehicle charging navigation strategy in coupled smart grid and transportation network: A hierarchical reinforcement learning approach

Battery and Hydrogen Energy Storage Control in a Smart Energy Network with Flexible Energy Demand Using Deep Reinforcement Learning

Longevity-conscious energy management strategy of fuel cell hybrid electric Vehicle Based on deep reinforcement learning

A Double‐Deep Q‐Network‐Based Energy Management Strategy for Hybrid Electric Vehicles under Variable Driving Cycles

Electric Vehicles Management in Distribution Network: A Data-Efficient Bi-level Safe Deep Reinforcement Learning Method

A Cooperative Dispatch Algorithm for Hydrogen-Based Grid-Connection Microgrids: A Multi-Agent Reinforcement Learning Method