Abstract:The random charging and dynamic traveling behaviors of massive plug-in electric vehicles (PEVs) pose challenges to the efficient and safe operation of transportation-electrification coupled systems (TECSs). To realize real-time scheduling of urban PEV fleet charging demand, this paper proposes a PEV decision-making guidance (PEVDG) strategy based on the bi-level deep reinforcement learning, achieving the reduction of user charging costs while ensuring the stable operation of distribution networks (DNs). For the discrete time-series characteristics and the heterogeneity of decision actions, the FEVDG problem is duly decoupled into a bi-level finite Markov decision process, in which the upper-lower layers are used respectively for charging station (CS) recommendation and path navigation. Specifically, the upper-layer agent realizes the mapping relationship between the environment state and the optimal CS by perceiving the PEV charging requirements, CS equipment resources and DN operation conditions. And the action decision output of the upper-layer is embedded into the state space of the lower-layer agent. Meanwhile, the lower-level agent determines the optimal road segment for path navigation by capturing the real-time PEV state and the transportation network information. Further, two elaborate reward mechanisms are developed to motivate and penalize the decision-making learning of the dual agents. Then two extension mechanisms (i.e., dynamic adjustment of learning rates and adaptive selection of neural network units) are embedded into the Rainbow algorithm based on the DQN architecture, constructing a modified Rainbow algorithm as the solution to the concerned bi-level decision-making problem. The average rewards for the upper-lower levels are ¥ -90.64 and ¥ 13.24 respectively. The average equilibrium degree of the charging service and average charging cost are 0.96 and ¥ 42.45, respectively. Case studies are conducted within a practical urban zone with the TECS. Extensive experimental results show that the proposed methodology improves the generalization and learning ability of dual agents, and facilitates the collaborative operation of traffic and electrical networks.

Prediction and Optimization of Pure Electric Vehicle Tire/road Structure-Borne Noise Based on Knowledge Graph and Multi-Task ResNet

Multitarget Prediction and Optimization of Pure Electric Vehicle Tire/road Airborne Noise Sound Quality Based on a Knowledge- and Data-Driven Method

Multi-service Provision for Electric Vehicles in Power-Transportation Networks Towards a Low-Carbon Transition: A Hierarchical and Hybrid Multi-Agent Reinforcement Learning Approach

Pure Electric Vehicle Nonstationary Interior Sound Quality Prediction Based on Deep CNNs with an Adaptable Learning Rate Tree

Multi-Objective Prediction and Optimization of Vehicle Acoustic Package Based on ResNet Neural Network

An Early Warning Protection Method for Electric Vehicle Charging Based on the Hybrid Neural Network Model

Pedestrian Trajectory Prediction and Pose Estimation Considering Behavioral Characteristic Relationships and Noise Inhibition

A hybrid electric vehicle load classification and forecasting approach based on GBDT algorithm and temporal convolutional network

Bi-level deep reinforcement learning for PEV decision-making guidance by coordinating transportation-electrification coupled systems

Jointly Dampening Traffic Oscillations and Improving Energy Consumption with Electric, Connected and Automated Vehicles: A Reinforcement Learning Based Approach

Design of optimal control strategy for range extended electric vehicles considering additional noise, vibration and harshness constraints

Deep Spatio-Temporal Residual Networks for Connected Urban Vehicular Traffic Prediction.

Estimating Urban Noise along Road Network from Street View Imagery

A Hybrid Numerical-Experimental Analysis for Tire Air-Pumping Noise with Application to Pattern Optimization

Adaptive Graph Convolutional Recurrent Network with Transformer and Whale Optimization Algorithm for Traffic Flow Prediction

Enhancing Traffic Flow Forecasting with Deep‐Learning and Noise Reduction Techniques

A Multi-Task Learning Network With a Collision-Aware Graph Transformer for Traffic-Agents Trajectory Prediction

Enhancing Demand Prediction: A Multi-Task Learning Approach for Taxis and TNCs

Transformer-Based Traffic-Aware Predictive Energy Management of a Fuel Cell Electric Vehicle

Multitask Learning and GCN-Based Taxi Demand Prediction for a Traffic Road Network

Context‐aware trajectory prediction for autonomous driving in heterogeneous environments