Abstract:Leveraging sensory information to aid the millimeter-wave (mmWave) and sub-terahertz (sub-THz) beam selection process is attracting increasing interest. This sensory data, captured for example by cameras at the basestations, has the potential of significantly reducing the beam sweeping overhead and enabling highly-mobile applications. The solutions developed so far, however, have mainly considered single-candidate scenarios, i.e., scenarios with a single candidate user in the visual scene, and were evaluated using synthetic datasets. To address these limitations, this paper extensively investigates the sensing-aided beam prediction problem in a real-world multi-object vehicle-to-infrastructure (V2I) scenario and presents a comprehensive machine learning-based framework. In particular, this paper proposes to utilize visual and positional data to predict the optimal beam indices as an alternative to the conventional beam sweeping approaches. For this, a novel user (transmitter) identification solution has been developed, a key step in realizing sensing-aided multi-candidate and multi-user beam prediction solutions. The proposed solutions are evaluated on the large-scale real-world DeepSense $6$G dataset. Experimental results in realistic V2I communication scenarios indicate that the proposed solutions achieve close to $100\%$ top-5 beam prediction accuracy for the scenarios with single-user and close to $95\%$ top-5 beam prediction accuracy for multi-candidate scenarios. Furthermore, the proposed approach can identify the probable transmitting candidate with more than $93\%$ accuracy across the different scenarios. This highlights a promising approach for nearly eliminating the beam training overhead in mmWave/THz communication systems.

Multimodal Deep Learning Empowered Millimeter-Wave Beam Prediction

Enhancing Mmwave Beam Prediction Through Deep Learning with Sub-6 GHz Channel Estimate

A Deep Learning-Based Spatial Domain Beam Prediction in Massive MIMO System

Deep Learning Assisted Mmwave Beam Prediction with Prior Low-frequency Information

Deep Learning-Based Radar-Assisted Beam Prediction

Deep Learning Based Spatio-temporal Mmwave Beam Prediction with Hierarchical Feature Mixing

Efficient Millimeter-Wave Beam Prediction: an WiFi Sensing-Assisted Deep Learning Approach

Camera Based mmWave Beam Prediction: Towards Multi-Candidate Real-World Scenarios

Advancing Multi-Modal Beam Prediction with Multipath-Like Data Augmentation and Efficient Fusion Mechanism

Vision and Radar Multimodal Aided Beam Prediction: Facilitating Metaverse Development

Vision-Position Multi-Modal Beam Prediction Using Real Millimeter Wave Datasets

Deep Learning Assisted Beam Prediction Using Out-of-Band Information.

Deep Learning Assisted mmWave Beam Prediction for Heterogeneous Networks: A Dual-Band Fusion Approach

Radar Aided 6G Beam Prediction: Deep Learning Algorithms and Real-World Demonstration

Joint Optimization of Data- and Model-Driven Probing Beams and Beam Predictor

Deep Learning Based Predictive Beamforming Design

Multi-Source Low Redundancy Data-Aided Beam Prediction for V2I Communication

Beam Tracking Based on Multi-Point Radar Sensing and DNN Prediction.

Beam prediction and tracking mechanism with enhanced LSTM for mmWave aerial base station

Beam Prediction based on Large Language Models

Beam Prediction for Mmwave Massive MIMO Using Adjustable Feature Fusion Learning.