Abstract:Reinforcement learning has been adopted to improve the efficiency of vehicular lane-changing (LC) decisions. However, since the local vehicular data needs to be uploaded to the edge node for accomplishing the learning and decision tasks, the data and traffic safety are under critical threat. Illegal data usage or attacks can generate misleading LC decisions such as collisions or rollover, which severely degrades the traffic safety. Therefore, this paper investigates a blockchain-FRL (federated reinforcement learning) approach for the LC decisions, which jointly accounts for the traffic, data and training safety of the LC decision-making. A vehicular reputation model based on the Bayesian risk situation is constructed and combined with the FRL. The proposed model can help evaluate the traffic safety, select the vehicles for participating in the FRL training, and adjust the FRL aggregation weights. The FRL can protect the data safety by enabling the exchange and aggregation of the LC decision network parameters. The blockchain can ensure the FRL training safety by recording the FRL task information. In addition, a Proof of Work (PoW) consensus scheme is devised to increase the FRL robustness, where the vehicles can collaboratively join the blockchain consensus and accomplish the FRL aggregation in a distributed manner. Two typical scenarios are selected for the experimental evaluation, including the highway scenario and the merging scenario. The experiment results indicate that the proposed method shows better efficiency, convergence and message safety delivery ratio by comparing with the existing studies.

Blockchain-FRL for Vehicular Lane-Changing: Towards Traffic, Data and Training Safety