Abstract:In this paper, the problem of smart grid energy management under stochastic dynamics is investigated. In the considered model, at the demand side, it is assumed that customers can act as prosumers who own renewable energy sources and can both produce and consume energy. Due to the coupling between the prosumers' decisions and the stochastic nature of renewable energy, the interaction among prosumers is formulated as a stochastic game, in which each prosumer seeks to maximize its payoff, in terms of revenues, by controlling its energy consumption and demand. In particular, the subjective behavior of prosumers is explicitly reflected into their payoff functions using prospect theory, a powerful framework that allows modeling real-life human choices. For this prospect-based stochastic game, it is shown that there always exists a stationary Nash equilibrium where the prosumers' trading policies in the equilibrium are independent of the time and their histories of the play. Moreover, a novel distributed algorithm with no information sharing among prosumers is proposed and shown to converge to an $\epsilon$-Nash equilibrium. On the other hand, at the supply side, the interaction between the utility company and the prosumers is formulated as an online optimization problem in which the utility company's goal is to learn its optimal energy allocation rules. For this case, it is shown that such an optimization problem admits a no-regret algorithm meaning that regardless of the actual outcome of the game among the prosumers, the utility company can follow a strategy that mitigates its allocation costs as if it knew the entire demand market a priori. Simulation results show the convergence of the proposed algorithms to their predicted outcomes and present new insights resulting from prospect theory that contribute toward more efficient energy management in the smart grids.

A proximal policy optimization based intelligent home solar management

Learning-based scheduling of industrial hybrid renewable energy systems

Proximal Policy Optimization for Energy Management of Electric Vehicles and PV Storage Units

Multi-Layer Energy Management and Strategy Learning for Microgrids: A Proximal Policy Optimization Approach

Online Optimization for Real-Time Peer-to-Peer Electricity Market Mechanisms

A Hybrid Approach for Home Energy Management with Imitation Learning and Online Optimization

Optimal Energy Management of a Residential Prosumer: A Robust Data-Driven Dynamic Programming Approach

Peer-to-Peer Energy Transactions for Prosumers Based on Improved Deep Deterministic Policy Gradient Algorithm

Proximal policy optimization through a deep reinforcement learning framework for remedial action schemes of VSC-HVDC

Deep Reinforcement Learning for Real-Time Energy Management in Smart Home

A Machine Learning Approach for Prosumer Management in Intraday Electricity Markets

Efficient Real-Time Residential Energy Management Through Milp Based Rolling Horizon Optimization

Optimization strategy based on deep reinforcement learning for home energy management

Robust Energy Management Policies for Solar Microgrids via Reinforcement Learning

Designing Near-Optimal Policies for Energy Management in a Stochastic Environment

Proximal Policy Optimization Algorithms

Beyond the Boundaries of Proximal Policy Optimization

Balancing Urban Energy Considering Economic Growth and Environmental Sustainability Through Integration of Renewable Energy

Stochastic Games for Smart Grid Energy Management with Prospect Prosumers

Online Optimal Power Scheduling of a Microgrid via Imitation Learning

Intelligent Real-Time Utilization of Hybrid Energy Resources for Cost Optimization in Smart Microgrids