Abstract:This paper proposes the integrated sensing and communication beam tracking based on deep reinforcement learning (IBTD) algorithm. To achieve accurate beam tracking and optimize the transmit power, we integrate the sensing results, such as the position and velocity of the target vehicle, along with the buffer pool status information, with deep reinforcement learning to select an appropriate policy. Simulation results demonstrate that the proposed IBTD algorithm achieves lower packet age and transmit power consumption compared to the baseline algorithms. Beam tracking is commonly employed in millimetre wave (mmWave) based vehicle‐to‐vehicle (V2V) networks to align the beams towards the intended targets and compensate for the path loss of mmWave signals. To mitigate the high latency issue arising from the tracking processes, integrated sensing and communication (ISAC) technology leverages the echo signal to sense the motion parameters of the target, achieving low‐latency beam tracking without requiring pilot and uplink feedback. Existing studies mainly focus on utilizing ISAC for beam alignment to track the target, without integrating beam tracking with resource allocation. In this paper, we propose the ISAC beam tracking based on deep reinforcement learning (IBTD) algorithm to address this problem. Specifically, we introduce the concept of packet age to measure communication performance. To achieve accurate beam tracking and optimize the transmit power, we integrate the sensing results, such as the position and velocity of the target vehicle, along with the buffer pool status information, with deep reinforcement learning (DRL) to select an appropriate policy. Furthermore, we consider the effect of inter‐vehicle distance and incorporate the changing of tracking targets into the DRL‐based policy. Simulation results demonstrate that the proposed IBTD algorithm achieves lower packet age and transmit power consumption compared to the baseline algorithms.

Image-Based Beam Tracking with Deep Learning for Mmwave V2I Communication Systems

Deep learning-based beam alignment in mmWave vehicular networks

3-D LiDAR and GPS Aided Beam Tracking in Millimeter Wave Vehicular Communications

LSTM-Based Predictive Mmwave Beam Tracking Via Sub-6 GHz Channels for V2I Communications

Deep Learning for Hierarchical Beam Alignment in mmWave Communication Systems

IBTD: A novel ISAC beam tracking based on deep reinforcement learning for mmWave V2V networks

Deep Learning based Millimeter Wave Beam Tracking at Mobile User: Design and Experiment

Position Aware 60 GHz mmWave Beamforming for V2V Communications Utilizing Deep Learning

Predictive Beam Tracking with Cooperative Sensing for Vehicle-to-Infrastructure Communications.

Deep Learning Based Joint Beam Selection and Precoding Design for Mmwave Systems with Lens Arrays

Hierarchical Beam Alignment for Millimeter-Wave Communication Systems: A Deep Learning Approach

Machine Learning Based Mmwave Channel Tracking in Vehicular Scenario

Intelligent Beam Training with Deep Convolutional Neural Network in Mmwave Communications.

Deep Learning Aided Two-Stage Multi-Finger Beam Training in Millimeter-Wave Communication

Sensing-Assisted Beam Tracking in V2I Networks: Extended Target Case

A Deep Learning Approach to Location- and Orientation-aided 3D Beam Selection for mmWave Communications

Vehicle Cameras Guide mmWave Beams: Approach and Real-World V2V Demonstration

Vision-Aided Beam Tracking: Explore the Proper Use of Camera Images with Deep Learning

Camera Based mmWave Beam Prediction: Towards Multi-Candidate Real-World Scenarios

Third-party Camera Aided Beam Alignment Real-world Prototype for Mmwave Communications

Beam Training in mmWave Vehicular Systems: Machine Learning for Decoupling Beam Selection