P2P Energy Trading through Prospect Theory, Differential Evolution, and Reinforcement Learning

Ashutosh Timilsina,Simone Silvestri

DOI: https://doi.org/10.1145/3603148

2023-06-10

ACM Transactions on Evolutionary Learning and Optimization

Abstract:Peer-to-peer (P2P) energy trading is a decentralized energy market where local energy prosumers act as peers, trading energy among each other. Existing works in this area largely overlook the importance of user behavioral modeling, assume users’ sustained active participation, and full compliance in the decision-making process. To overcome these unrealistic assumptions, and their deleterious consequences, in this paper we propose an automated P2P energy trading framework that specifically considers the users’ perception by exploiting prospect theory . We formalize an optimization problem that maximizes the buyers’ perceived utility while matching energy production and demand. We prove that the problem is NP-hard and we propose a Differential Evolution-based Algorithm for Trading Energy ( DEbATE ) heuristic. Additionally, we propose two automated pricing solutions to improve the sellers’ profit based on reinforcement learning. The first solution, named Pricing mechanism with Q-learning and Risk-sensitivity ( PQR ), is based on Q-learning. Additionally, the given scalability issues of PQR , we propose a Deep Q-Network-based algorithm called ProDQN that exploits deep learning and a novel loss function rooted in prospect theory. Results based on real traces of energy consumption and production, as well as realistic prospect theory functions, show that our approaches achieve \(26\% \) higher perceived value for buyers and generate \(7\% \) more reward for sellers, compared to recent state-of-the-art approaches.