Abstract:Advancements in 6G wireless technology have elevated the importance of beamforming, especially for attaining ultra-high data rates via millimeter-wave (mmWave) frequency deployment. Although promising, mmWave bands require substantial beam training to achieve precise beamforming. While initial deep learning models that use RGB camera images demonstrated promise in reducing beam training overhead, their performance suffers due to sensitivity to lighting and environmental variations. Due to this sensitivity, Quality of Service (QoS) fluctuates, eventually affecting the stability and dependability of networks in dynamic environments. This emphasizes a critical need for robust solutions. This paper proposes a robust beamforming technique to ensure consistent QoS under varying environmental conditions. An optimization problem has been formulated to maximize users' data rates. To solve the formulated NP-hard optimization problem, we decompose it into two subproblems: the semantic localization problem and the optimal beam selection problem. To solve the semantic localization problem, we propose a novel method that leverages the K-means clustering and YOLOv8 model. To solve the beam selection problem, we propose a novel lightweight hybrid architecture that combines a lightweight transformer with a CNN architecture through a weighted entropy mechanism. This hybrid architecture utilizes multimodal data sources to dynamically predict the optimal beams. A novel metric, Accuracy-Complexity Efficiency (ACE), has been proposed to quantify this. Six testing scenarios have been developed to evaluate the robustness of the proposed model. Finally, the simulation result demonstrates that the proposed model outperforms several state-of-the-art baselines regarding beam prediction accuracy, received power, and ACE in the developed test scenarios.

Vision Aided Environment Semantics Extraction and Its Application in Mmwave Beam Selection

3D Scene Based Beam Selection for Mmwave Communications

Environment Semantics Aided Wireless Communications: A Case Study of mmWave Beam Prediction and Blockage Prediction

Enhancing Mmwave Beam Prediction Through Deep Learning with Sub-6 GHz Channel Estimate

Vision-Assisted mmWave Beam Management for Next-Generation Wireless Systems: Concepts, Solutions and Open Challenges

Multi-View Millimeter-Wave Imaging over Wireless Cellular Network

Environment Sensing with Beam Sweeping and Non-Uniform Pixelation in Wireless Communication Systems

Camera Based mmWave Beam Prediction: Towards Multi-Candidate Real-World Scenarios

Environment Semantic Communication: Enabling Distributed Sensing Aided Networks

Proactive Base Station Selection Empowered by Multi-View Images

Multimodal Deep Learning Empowered Millimeter-Wave Beam Prediction

Environment Sensing Considering the Occlusion Effect: A Multi-View Approach

Multi-Camera Views Based Beam Searching and BS Selection with Reduced Training Overhead

Vision-Aided Beam Tracking: Explore the Proper Use of Camera Images with Deep Learning

Vision-Position Multi-Modal Beam Prediction Using Real Millimeter Wave Datasets

Third-party Camera Aided Beam Alignment Real-world Prototype for Mmwave Communications

A Deep Learning Approach to Location- and Orientation-aided 3D Beam Selection for mmWave Communications

Multi-Camera View Based Proactive BS Selection and Beam Switching for V2X

Advancing Ultra-Reliable 6G: Transformer and Semantic Localization Empowered Robust Beamforming in Millimeter-Wave Communications

How to Define the Propagation Environment Semantics and Its Application in Scatterer-Based Beam Prediction

Hybrid Knowledge-Data Driven Channel Semantic Acquisition and Beamforming for Cell-Free Massive MIMO