Abstract:This paper proposes the integrated sensing and communication beam tracking based on deep reinforcement learning (IBTD) algorithm. To achieve accurate beam tracking and optimize the transmit power, we integrate the sensing results, such as the position and velocity of the target vehicle, along with the buffer pool status information, with deep reinforcement learning to select an appropriate policy. Simulation results demonstrate that the proposed IBTD algorithm achieves lower packet age and transmit power consumption compared to the baseline algorithms. Beam tracking is commonly employed in millimetre wave (mmWave) based vehicle‐to‐vehicle (V2V) networks to align the beams towards the intended targets and compensate for the path loss of mmWave signals. To mitigate the high latency issue arising from the tracking processes, integrated sensing and communication (ISAC) technology leverages the echo signal to sense the motion parameters of the target, achieving low‐latency beam tracking without requiring pilot and uplink feedback. Existing studies mainly focus on utilizing ISAC for beam alignment to track the target, without integrating beam tracking with resource allocation. In this paper, we propose the ISAC beam tracking based on deep reinforcement learning (IBTD) algorithm to address this problem. Specifically, we introduce the concept of packet age to measure communication performance. To achieve accurate beam tracking and optimize the transmit power, we integrate the sensing results, such as the position and velocity of the target vehicle, along with the buffer pool status information, with deep reinforcement learning (DRL) to select an appropriate policy. Furthermore, we consider the effect of inter‐vehicle distance and incorporate the changing of tracking targets into the DRL‐based policy. Simulation results demonstrate that the proposed IBTD algorithm achieves lower packet age and transmit power consumption compared to the baseline algorithms.

Beam management optimization for V2V communications based on deep reinforcement learning

Deep Reinforcement Learning Assisted Beam Tracking and Data Transmission for 5G V2X Networks

Deep Reinforcement Learning Assisted Hybrid Precoding for V2I Communications with Doppler Shift and Delay Spread

Efficient Initial Access with Deep Reinforcement Learning Based Beam Sweeping in Wireless Cellular Communication Systems

Joint Relay Selection and Beam Management Based on Deep Reinforcement Learning for Millimeter Wave Vehicular Communication

Learning-assisted User Scheduling and Beamforming for mmWave Vehicular Networks

Reinforcement Learning-Based Joint Beamwidth and Beam Alignment Interval Optimization in V2I Communications

Intelligent Vehicle Network Routing With Adaptive 3D Beam Alignment for mmWave 5G-Based V2X Communications

Attention-deep reinforcement learning jointly beamforming based on tensor decomposition for RIS-assisted V2X mmWave massive MIMO system

Energy-Efficient Resource Allocation Based on Deep Q-Network in V2V Communications

Hybrid Beamforming for RIS-Empowered Multi-hop Terahertz Communications: A DRL-based Method

Machine Learning Enabling Analog Beam Selection for Concurrent Transmissions in Millimeter-Wave V2V Communications

IBTD: A novel ISAC beam tracking based on deep reinforcement learning for mmWave V2V networks

Deep Reinforcement Learning-Based Coordinated Beamforming for mmWave Massive MIMO Vehicular Networks

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Deep Reinforcement Learning for 5G Networks: Joint Beamforming, Power Control, and Interference Coordination

Position Aware 60 GHz mmWave Beamforming for V2V Communications Utilizing Deep Learning

Joint Sensing and Communications for Deep Reinforcement Learning-based Beam Management in 6G

Joint Communication and Control for mmWave/THz Beam Alignment in V2X Networks

A Belief Propagation Solution for Beam Coordination in mmWave Vehicular Networks

Reinforcement Learning Based Vehicle-cell Association Algorithm for Highly Mobile Millimeter Wave Communication