Abstract:This paper proposes the integrated sensing and communication beam tracking based on deep reinforcement learning (IBTD) algorithm. To achieve accurate beam tracking and optimize the transmit power, we integrate the sensing results, such as the position and velocity of the target vehicle, along with the buffer pool status information, with deep reinforcement learning to select an appropriate policy. Simulation results demonstrate that the proposed IBTD algorithm achieves lower packet age and transmit power consumption compared to the baseline algorithms. Beam tracking is commonly employed in millimetre wave (mmWave) based vehicle‐to‐vehicle (V2V) networks to align the beams towards the intended targets and compensate for the path loss of mmWave signals. To mitigate the high latency issue arising from the tracking processes, integrated sensing and communication (ISAC) technology leverages the echo signal to sense the motion parameters of the target, achieving low‐latency beam tracking without requiring pilot and uplink feedback. Existing studies mainly focus on utilizing ISAC for beam alignment to track the target, without integrating beam tracking with resource allocation. In this paper, we propose the ISAC beam tracking based on deep reinforcement learning (IBTD) algorithm to address this problem. Specifically, we introduce the concept of packet age to measure communication performance. To achieve accurate beam tracking and optimize the transmit power, we integrate the sensing results, such as the position and velocity of the target vehicle, along with the buffer pool status information, with deep reinforcement learning (DRL) to select an appropriate policy. Furthermore, we consider the effect of inter‐vehicle distance and incorporate the changing of tracking targets into the DRL‐based policy. Simulation results demonstrate that the proposed IBTD algorithm achieves lower packet age and transmit power consumption compared to the baseline algorithms.

Image and index fused sequence-to-sequence algorithm for vision-aided millimeter-wave beam tracking

Vision-Aided Beam Tracking: Explore the Proper Use of Camera Images with Deep Learning

Multi-modal fusion for sensing-aided beam tracking in mmWave communications

Millimetre-Wave Beam Tracking: an Intelligent Machine Learning and Kalman Filter Fusion Technology

Vision-aided Multi-user Beam Tracking for Mmwave Massive MIMO System: Prototyping and Experimental Results

Beam Selection for RIS-Enabled Terahertz Multi-User MIMO Systems via Multi-Task Learning

Camera Based mmWave Beam Prediction: Towards Multi-Candidate Real-World Scenarios

Deep Learning based Computer-vision for Enhanced Beamforming

Vehicle Cameras Guide mmWave Beams: Approach and Real-World V2V Demonstration

Multimodal Fusion Assisted mmWave Beam Training in Dual-Model Networks

Vision-Position Multi-Modal Beam Prediction Using Real Millimeter Wave Datasets

Enhanced Target Tracking Algorithm for Autonomous Driving Based on Visible and Infrared Image Fusion

Multimodal Deep Learning Empowered Millimeter-Wave Beam Prediction

Beam Prediction for Mmwave Massive MIMO Using Adjustable Feature Fusion Learning.

Deep Learning Aided Two-Stage Multi-Finger Beam Training in Millimeter-Wave Communication

IBTD: A novel ISAC beam tracking based on deep reinforcement learning for mmWave V2V networks

TIME-SEQUENCE CHANNEL INFERENCE FOR BEAM ALIGNMENT IN VEHICULAR NETWORKS

Integrated Sensing and Communications Towards Proactive Beamforming in mmWave V2I via Multi-Modal Feature Fusion (MMFF)

Channel Estimation and Hybrid Precoding for Millimeter Wave Communications: A Deep Learning-Based Approach

Machine Learning-Based mmWave MIMO Beam Tracking in V2I Scenarios: Algorithms and Datasets

Intelligent Angle Map-based Beam Alignment for RIS-aided mmWave Communication Networks