Abstract:As the energy landscape evolves toward sustainability, the accelerating integration of distributed energy resources poses challenges to the operability and reliability of the electricity grid. One significant aspect of this issue is the notable increase in net load variability at the grid edge. Transactive energy, implemented through local energy markets, has recently garnered attention as a promising solution to address the grid challenges in the form of decentralized, indirect demand response on a community level. Given the nature of these challenges, model-free control approaches, such as deep reinforcement learning, show promise for the decentralized automation of participation within this context. Existing studies at the intersection of transactive energy and model-free control primarily focus on socioeconomic and self-consumption metrics, overlooking the crucial goal of reducing community-level net load variability. This study addresses this gap by training a set of deep reinforcement learning agents to automate end-user participation in ALEX, an economy-driven local energy market. In this setting, agents do not share information and only prioritize individual bill optimization. The study unveils a clear correlation between bill reduction and reduced net load variability in this setup. The impact on net load variability is assessed over various time horizons using metrics such as ramping rate, daily and monthly load factor, as well as daily average and total peak export and import on an open-source dataset. Agents are then benchmarked against several baselines, with their performance levels showing promising results, approaching those of a near-optimal dynamic programming benchmark.

Indirect Multi-energy Transactions of Energy Internet with Deep Reinforcement Learning Approach

Integrated Demand Response for a Load Serving Entity in Multi-Energy Market Considering Network Constraints

Optimal Energy Operation Strategy for We-Energy of Energy Internet Based on Hybrid Reinforcement Learning With Human-in-the-Loop

An Integrated Demand Response-Based Energy Management Strategy for Integrated Energy System Based on Deep Reinforcement Learning

Price-Matching-Based Regional Energy Market with Hierarchical Reinforcement Learning Algorithm

Optimal Energy Management Strategies for Energy Internet Via Deep Reinforcement Learning Approach

Multi-Agent Deep Reinforcement Learning for Coordinated Energy Trading and Flexibility Services Provision in Local Electricity Markets

Peer-to-Peer Energy Trading and Energy Conversion in Interconnected Multi-Energy Microgrids Using Multi-Agent Deep Reinforcement Learning

Peer-to-peer energy trading with energy trading consistency in interconnected multi-energy microgrids: A multi-agent deep reinforcement learning approach

A Hierarchical Deep Reinforcement Learning-Based Community Energy Trading Scheme for a Neighborhood of Smart Households

Deep Reinforcement Learning Based Optimal Energy Management of Multi-energy Microgrids with Uncertainties

Multi-agent energy management optimization for integrated energy systems under the energy and carbon co-trading market

Multi-agent deep reinforcement learning based distributed control architecture for interconnected multi-energy microgrid energy management and optimization

On-line Building Energy Optimization using Deep Reinforcement Learning

Multi-Agent Deep Reinforcement Learning for Blockchain-Based Energy Trading in Decentralized Electric Vehicle Charger-Sharing Networks

Mean-Field Multi-Agent Reinforcement Learning for Peer-to-Peer Multi-Energy Trading

Decentralized Coordination of Distributed Energy Resources through Local Energy Markets and Deep Reinforcement Learning

Reinforcement Learning-Based Energy Trading and Management of Regional Interconnected Microgrids

Federated reinforcement learning for smart building joint peer-to-peer energy and carbon allowance trading

Transactive Energy Supported Economic Operation for Multi-Energy Complementary Microgrids

A Multi-Agent Deep Reinforcement Learning Approach for a Distributed Energy Marketplace in Smart Grids