Abstract:This paper proposes the integrated sensing and communication beam tracking based on deep reinforcement learning (IBTD) algorithm. To achieve accurate beam tracking and optimize the transmit power, we integrate the sensing results, such as the position and velocity of the target vehicle, along with the buffer pool status information, with deep reinforcement learning to select an appropriate policy. Simulation results demonstrate that the proposed IBTD algorithm achieves lower packet age and transmit power consumption compared to the baseline algorithms. Beam tracking is commonly employed in millimetre wave (mmWave) based vehicle‐to‐vehicle (V2V) networks to align the beams towards the intended targets and compensate for the path loss of mmWave signals. To mitigate the high latency issue arising from the tracking processes, integrated sensing and communication (ISAC) technology leverages the echo signal to sense the motion parameters of the target, achieving low‐latency beam tracking without requiring pilot and uplink feedback. Existing studies mainly focus on utilizing ISAC for beam alignment to track the target, without integrating beam tracking with resource allocation. In this paper, we propose the ISAC beam tracking based on deep reinforcement learning (IBTD) algorithm to address this problem. Specifically, we introduce the concept of packet age to measure communication performance. To achieve accurate beam tracking and optimize the transmit power, we integrate the sensing results, such as the position and velocity of the target vehicle, along with the buffer pool status information, with deep reinforcement learning (DRL) to select an appropriate policy. Furthermore, we consider the effect of inter‐vehicle distance and incorporate the changing of tracking targets into the DRL‐based policy. Simulation results demonstrate that the proposed IBTD algorithm achieves lower packet age and transmit power consumption compared to the baseline algorithms.

LSTM-Based Predictive Mmwave Beam Tracking Via Sub-6 GHz Channels for V2I Communications

Enhancing Mmwave Beam Prediction Through Deep Learning with Sub-6 GHz Channel Estimate

Machine Learning Based Mmwave Channel Tracking in Vehicular Scenario

Beam prediction and tracking mechanism with enhanced LSTM for mmWave aerial base station

LSTM-Aided Selective Beam Tracking in Multi-Cell Scenario for mmWave Wireless Systems

Image-Based Beam Tracking with Deep Learning for Mmwave V2I Communication Systems

Position Prediction Based Fast Beam Tracking Scheme for Multi-User UAV-mmWave Communications.

FC-BET: A Fast Consecutive Beam Tracking Scheme for MmWave Vehicular Communications

KF-LSTM Based Beam Tracking for UAV-Assisted Mmwave HSR Wireless Networks.

Mobility-aware Adaptive Beam Tracking for Vehicles in MmWave Communication Networks

3-D LiDAR and GPS Aided Beam Tracking in Millimeter Wave Vehicular Communications

Camera Based mmWave Beam Prediction: Towards Multi-Candidate Real-World Scenarios

Predictive Beam Tracking with Cooperative Sensing for Vehicle-to-Infrastructure Communications.

Integrated Sensing and Communication-Enabled Predictive Beamforming with Deep Learning in Vehicular Networks

Sensing-Assisted Beam Tracking in V2I Networks: Extended Target Case

Machine Learning Prediction for Phase-less Millimeter-Wave Beam Tracking

Location-Based Beam Tracking and Doppler-Aware Vehicle Scheduling for V2I Communications

A Beam Tracking Scheme Based on Deep Reinforcement Learning for Multiple Vehicles.

IBTD: A novel ISAC beam tracking based on deep reinforcement learning for mmWave V2V networks

Predictive hierarchical beam training with noisy ranging measurements for mmWave vehicular communications

Autonomous Self-Trained Channel State Prediction Method for mmWave Vehicular Communications