Abstract:The connected autonomous vehicle is con-sidered an effective way to improve transport safety and efficiency.To overcome the limited sensing and computing capabilities of individual vehicles,we de-sign a digital twin assisted decision-making frame-work for Internet of Vehicles,by leveraging the in-tegration of communication,sensing and computing.In this framework,the digital twin entities residing on edge can effectively communicate and cooperate with each other to plan sub-targets for their respec-tive vehicles,while the vehicles only need to achieve the sub-targets by generating a sequence of atomic ac-tions.Furthermore,we propose a hierarchical multi-agent reinforcement learning approach to implement the framework,which can be trained in an end-to-end way.In the proposed approach,the communication interval of digital twin entities could adapt to time-varying environment.Extensive experiments on driv-ing decision-making have been performed in traffic junction scenarios of different difficulties.The exper-imental results show that the proposed approach can largely improve collaboration efficiency while reduc-ing communication overhead.

Communication-Efficient Decision-Making of Digital Twin Assisted Internet of Vehicles:A Hierarchical Multi-Agent Reinforcement Learning Approach