Abstract:In the context of firefighting and rescue operations, a cross-modal semantic segmentation model based on a single-chip millimeter-wave (mmWave) radar for indoor environmental perception is proposed and discussed. To efficiently obtain high-quality labels, an automatic label generation method utilizing LiDAR point clouds and occupancy grid maps is introduced. The proposed segmentation model is based on U-Net. A spatial attention module is incorporated, which enhanced the performance of the mode. The results demonstrate that cross-modal semantic segmentation provides a more intuitive and accurate representation of indoor environments. Unlike traditional methods, the model's segmentation performance is minimally affected by azimuth. Although performance declines with increasing distance, this can be mitigated by a well-designed model. Additionally, it was found that using raw ADC data as input is ineffective; compared to RA tensors, RD tensors are more suitable for the proposed model.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to solve the key challenges in indoor environmental perception, especially in the fire rescue scenario. Specifically, the author proposes a cross - modal semantic segmentation model based on single - chip millimeter - wave (mmWave) radar to achieve efficient perception of the indoor environment. The following are the main problems that this paper attempts to solve: 1. **Limitations of traditional sensors in fire scenarios**: - In smoky and dim - lit environments, cameras cannot expose and capture images correctly. - Lidar (LiDAR) will be affected by scattering in smoke, resulting in a decline in point - cloud quality. 2. **Application potential of millimeter - wave radar**: - Millimeter - wave radar shows strong robustness in harsh conditions such as smoke and thick fog, so it is especially suitable for environmental perception in fire scenarios. - However, single - chip millimeter - wave radar has challenges in generating high - quality point clouds due to hardware limitations and insufficient signal processing methods. 3. **Improving the accuracy and efficiency of environmental perception**: - A semantic segmentation model based on the U - Net architecture is proposed, and a spatial attention module (Spatial Attention Module) is introduced to enhance the model performance. - To reduce the time and cost of manual annotation, an automatic label generation method based on LiDAR point clouds and occupancy grid maps is proposed. 4. **Meeting the challenges of complex indoor environments**: - Indoor environments are more complex than outdoor environments, which place higher requirements on cross - modal models. - Designing reasonable segmentation tasks and labels is the key to improving model performance. 5. **Verifying the feasibility of single - chip millimeter - wave radar**: - The feasibility of single - chip millimeter - wave radar in indoor environmental perception is verified, and its adaptability under different input data types is demonstrated. ### Summary By proposing a cross - modal semantic segmentation model based on single - chip millimeter - wave radar, this paper aims to solve the limitations of traditional sensors in fire scenarios and improve the accuracy and efficiency of indoor environmental perception. At the same time, by introducing an automatic label generation method and a spatial attention module, the performance and practicality of the model are further enhanced.

Cross-modal semantic segmentation for indoor environmental perception using single-chip millimeter-wave radar raw data

Radar Style Transfer for Metric Robot Localisation on Lidar Maps

Human Semantic Segmentation using Millimeter-Wave Radar Sparse Point Clouds

A Data-Driven Method for Indoor Radar Ghost Recognition with Environmental Mapping

RSS-Net: Weakly-Supervised Multi-Class Semantic Segmentation with FMCW Radar

An RGB-D Fusion Based Semantic Segmentation Algorithm Based on Neighborhood Metric Relations

Real-Time Semantic Segmentation of LiDAR Point Clouds on Edge Devices for Unmanned Systems

See Through Smoke: Robust Indoor Mapping with Low-cost mmWave Radar

Semantic Segmentation of Radar Detections using Convolutions on Point Clouds

SAR Image Segmentation Based on Hierarchical Visual Semantic and Adaptive Neighborhood Multinomial Latent Model

A Neural Network Based System for Efficient Semantic Segmentation of Radar Point Clouds

LiCROcc: Teach Radar for Accurate Semantic Occupancy Prediction using LiDAR and Camera

mmSignature: Semi-supervised human identification system based on millimeter wave radar

Deep Instance Segmentation with Automotive Radar Detection Points

Position-Feature Attention Network-Based Approach for Semantic Segmentation of Urban Building Point Clouds from Airborne Array Interferometric SAR

A Novel Radar Point Cloud Generation Method for Robot Environment Perception

Dense 3D Point Cloud Environmental Mapping Using Millimeter-Wave Radar

Learning Omni-Dimensional Spatio-Temporal Dependencies for Millimeter-Wave Radar Perception

Statistical methods for assessing agreement.

Enhancing mmWave Radar Point Cloud via Visual-inertial Supervision