mDS-PCGR: A Bimodal Gait Recognition Framework With the Fusion of 4-D Radar Point Cloud Sequences and Micro-Doppler Signatures

Chongrun Ma,Zhenyu Liu

DOI: https://doi.org/10.1109/jsen.2024.3355421

IF: 4.3

2024-03-15

IEEE Sensors Journal

Abstract:Radar-based gait recognition has emerged as a promising solution for noninvasive human identification. However, relying solely on single-modal radar gait representations, such as micro-Doppler signature and radar point cloud, proves inadequate for robust gait recognition in the presence of complex perceptual conditions. Additionally, achieving a high level of generalization, particularly when dealing with new subjects having limited training samples, is crucial for practical gait recognition. To address these challenges, we present a novel joint micro-Doppler and radar point clouds gait recognition (mDS-PCGR) framework in this study. This framework fuses gait features derived from both micro-Doppler signatures and 4-D radar point cloud sequences. First, a tracking-based preprocessing method is proposed to acquire high-quality micro-Doppler signatures and 4-D radar point cloud sequences, while suppressing the multipath interference in complex perceptual conditions. Second, a dual-flow fusion network is designed to extract discriminative gait features based on complementation of the two modalities to each other. Finally, a metric-based few-shot learning mechanism is used to instruct the optimization of dual-flow fusion network, combining triplet loss with center loss to achieve the identification of new subjects with few training samples. Extensive evaluation on real 4-D millimeter-wave radar measurement under multipath interfered and cross-view conditions is provided. Experimental results show the superior performance of the proposed mDS-PCGR, leveraging effective gait information from two modalities. It outperforms single-modal gait recognition methods and achieves the highest identification accuracy for new subjects with limited gallery samples.

engineering, electrical & electronic,instruments & instrumentation,physics, applied