Abstract:As the construction of supporting infrastructure for electric vehicles (EV) becomes more and more perfect, an energy replenishment station (ERS) involving photovoltaics (PV) that can provide charging and battery swapping services for electric vehicle owners comes into the vision of humanity. The operation optimization of each device in the ERS is conducive to improving the service capacity of the ERS, extending the service life of the energy storage batteries (ESB), and enhancing the economic benefits of the ERS. However, traditional model-based optimization algorithms cannot fully consider the stochastic nature of EV owners’ charging and battery swapping demands, the uncertainty of PV output, and the complex operating characteristics of ESB. Therefore, we propose a deep reinforcement learning-based adaptive optimal operation method for ERS considering ESB’s losses. Firstly, a mathematical model of each device in the ERS is established, and a refined energy storage model is introduced to describe ESB’s capacity degradation and efficiency decay. Secondly, to solve the dimensional disaster problem, the state space and action space selection method, and the charging strategy of batteries in the battery swapping station (BSS) are proposed to apply to the ERS, thus modeling the ERS optimization operation problem as a Markov decision process. Then, the solution is performed using a neural network-based proximal policy optimization (PPO) algorithm, consisting of a recurrent neural network that extracts information about the PV outflow trend and a deep neural network used to generate a control policy. Finally, the effectiveness of the proposed method is verified by simulation calculations, which not only enable adaptive decision-making under different PV output scenarios, but also consider the availability of EV battery swapping services, energy storage losses, and the economic benefits of the ERS.

Reinforcement Learning Driven Self-Optimizing Operation for Distributed Battery Electrical Storage System

Load Forecasting-Based Learning System for Energy Management With Battery Degradation Estimation: A Deep Reinforcement Learning Approach

Reinforcement Learning-Enhanced Adaptive Scheduling of Battery Energy Storage Systems in Energy Markets

Adaptive BESS and Grid Setpoints Optimization: A Model-Free Framework for Efficient Battery Management under Dynamic Tariff Pricing

Bi-level optimization of charging scheduling of a battery swap station based on deep reinforcement learning

Dynamic Balancing-Charging Management for Shared Autonomous Electric Vehicle Systems: A Two-Stage Learning-Based Approach

Reinforcement Learning-Based Scheduling of Multi-Battery Energy Storage System

Deep Reinforcement Learning Based Charging Scheduling for Household Electric Vehicles in Active Distribution Network

Optimal Electric Vehicle Charging Strategy with Markov Decision Process and Reinforcement Learning Technique

Reinforcement Learning for Battery Energy Storage Dispatch augmented with Model-based Optimizer

Developing Optimal Energy Arbitrage Strategy for Energy Storage System Using Reinforcement Learning

Mobile battery energy storage system control with knowledge‐assisted deep reinforcement learning

Operation of Distributed Battery Considering Demand Response Using Deep Reinforcement Learning in Grid Edge Control

Adaptive Optimization Operation of Electric Vehicle Energy Replenishment Stations Considering the Degradation of Energy Storage Batteries

Stochastic dispatch of energy storage in microgrids: An augmented reinforcement learning approach

Electric Vehicles Management in Distribution Network: A Data-Efficient Bi-level Safe Deep Reinforcement Learning Method

Branching Dueling Q-Network Based Online Scheduling of a Microgrid With Distributed Energy Storage Systems

An optimal solutions-guided deep reinforcement learning approach for online energy storage control

Stochastic Dispatch of Energy Storage in Microgrids: A Reinforcement Learning Approach Incorporated with MCTS

Voltage Regulation Based on Deep Reinforcement Learning Algorithm in Distribution Network with Energy Storage System

A Unified Configurational Optimization Framework for Battery Swapping and Charging Stations Considering Electric Vehicle Uncertainty