Real-Time Through-Wall Multiperson 3-D Pose Estimation Based on MIMO Radar

Changlong Wang,Dongsheng Zhu,Lijuan Sun,Chong Han,Jian Guo
DOI: https://doi.org/10.1109/tim.2024.3373058
IF: 5.6
2024-03-19
IEEE Transactions on Instrumentation and Measurement
Abstract:Detecting subsequent targets behind obstacles has always been a major challenge for humans. Based on the characteristic that low-frequency through-wall radar signals can penetrate obstacles and reflect on the target surface, we propose a real-time multiperson 3-D pose estimation method based on multiple-input multiple-output (MIMO) through-wall radar. Our proposed approach addresses the problem of being unable to recover human poses in real time when the human body is completely obstructed by an obstacle. This method follows a cross-modal supervision learning pipeline, first using a synchronized camera and a self-designed through-wall radar to obtain paired RGB images and through-wall radar images. Then, we use a monocular camera-based 3-D pose estimation method as a teacher network to extract 3-D poses from RGB images as a supervision signal. Finally, we design an end-to-end radar pose estimation network based on a transformer (ERPENT) as a student network, which learns pose features from two different temporal radar signals and directly predicts the final 3-D human pose. Unlike existing radar-based multiperson pose estimation methods, the proposed method in this article does not require an additional radar target reflection region detection process, thus greatly shortening the necessary pose estimation time. In practical applications, the 3-D human pose can be predicted by simply collecting through-wall radar images and inputting them into ERPENT. The experiment shows that under the condition of a 28-cm compact brick wall scene, the average positioning accuracy of human keypoints is 40.15 mm. The time cost of reconstructing a frame of the 3-D pose is 0.082 s.
engineering, electrical & electronic,instruments & instrumentation
What problem does this paper attempt to address?