Abstract:With the proliferation of electric vehicles (EVs), the transportation network and power grid become increasingly interdependent and coupled via charging stations. The concomitant growth in charging demand has posed challenges for both networks, highlighting the importance of charging coordination. Existing literature largely overlooks the interactions between power grid security and traffic efficiency. In view of this, we study the en-route charging station (CS) recommendation problem for EVs in dynamically coupled transportation-power systems. The system-level objective is to maximize the overall traffic efficiency while ensuring the safety of the power grid. This problem is for the first time formulated as a constrained Markov decision process (CMDP), and an online prediction-assisted safe reinforcement learning (OP-SRL) method is proposed to learn the optimal and secure policy by extending the PPO method. To be specific, we mainly address two challenges. First, the constrained optimization problem is converted into an equivalent unconstrained optimization problem by applying the Lagrangian method. Second, to account for the uncertain long-time delay between performing CS recommendation and commencing charging, we put forward an online sequence-to-sequence (Seq2Seq) predictor for state augmentation to guide the agent in making forward-thinking decisions. Finally, we conduct comprehensive experimental studies based on the Nguyen-Dupuis network and a large-scale real-world road network, coupled with IEEE 33-bus and IEEE 69-bus distribution systems, respectively. Results demonstrate that the proposed method outperforms baselines in terms of road network efficiency, power grid safety, and EV user satisfaction. The case study on the real-world network also illustrates the applicability in the practical context.

Dynamic Incentive Pricing on Charging Stations for Real-Time Congestion Management in Distribution Network: An Adaptive Model-Based Safe Deep Reinforcement Learning Method

Dynamic Coordinated Pricing for Coupled Power-Transportation Network

Deep Reinforcement Learning Based Charging Scheduling for Household Electric Vehicles in Active Distribution Network

Electric Vehicles Management in Distribution Network: A Data-Efficient Bi-level Safe Deep Reinforcement Learning Method

Dynamic Balancing-Charging Management for Shared Autonomous Electric Vehicle Systems: A Two-Stage Learning-Based Approach

Optimal Electric Vehicle Charging Strategy with Markov Decision Process and Reinforcement Learning Technique

Model-Free Real-Time EV Charging Scheduling Based on Deep Reinforcement Learning

Deep Reinforcement Learning-Based Strategy for Charging Station Participating in Demand Response

Deep Reinforcement Learning-Based Charging Price Determination Considering the Coordinated Operation of Hydrogen Fuel Cell Electric Vehicle, Power Network and Transportation Network

A Stochastic Game Approach for PEV Charging Station Operation in Smart Grid

Pricing model for the charging of electric vehicles based on system dynamics in Beijing

Deep Reinforcement Learning-Based Spatiotemporal Decision of Utility-Scale Highway Portable Energy Storage Systems

Multiagent Deep Reinforcement Learning for Electric Vehicle Fast Charging Station Pricing Game in Electricity-Transportation Nexus

Multi-agent reinforcement mechanism design for dynamic pricing-based demand response in charging network

Optimal Dynamic Pricing of Fast Charging Stations Considering Bounded Rationality of Users and Market Regulation

Deep Reinforcement Learning for Charging Scheduling of Electric Vehicles Considering Distribution Network Voltage Stability

Optimal Management for EV Charging Stations: A Win–Win Strategy for Different Stakeholders Using Constrained Deep Q-Learning

Online Prediction-Assisted Safe Reinforcement Learning for Electric Vehicle Charging Station Recommendation in Dynamically Coupled Transportation-Power Systems

EV Charging Command Fast Allocation Approach based on Deep Reinforcement Learning with Safety Modules

A deep reinforcement learning-based charging scheduling approach with augmented Lagrangian for electric vehicles

Bounded Rational Real‐time Charging Pricing Strategy under the Traffic‐grid Coupling Network