Multi-Modal Context Propagation for Person Re-Identification with Wireless Positioning

Yiheng Liu,Wengang Zhou,Mao Xi,Sanjing Shen,Houqiang Li
DOI: https://doi.org/10.1109/tmm.2021.3092579
IF: 7.3
2021-01-01
IEEE Transactions on Multimedia
Abstract:Existing person re-identification methods mainly rely on the visual appearance captured by cameras for identity matching. However, dueto the sensitivity of visual data to occlusion, blur, clothing change, etc., existing methods struggle to distinguish pedestrians in challenging scenarios. Inspired by the fact that most pedestrians carry around smart wireless devices, e.g. mobile phones that can be sensed by WiFi or cellular networks as wireless positioning signals, we propose to exploit the free yet informative wireless signals to assist person re-identification. It is well recognized that wireless signals are robust to visual noises mentioned above, which perform as a good complement to the visual data. To make full use of these multi-modal clues for person re-identification, we propose a multi-modal context propagation framework MCPF that contains a recurrent context propagation module RCPM and an unsupervised multi-modal cross-domain method UMM-ReID. RCPM enables context information to be continuously propagated and fused between visual data and wireless data. UMM-ReID utilizes wireless signals to constrain the estimation of pseudo labels. We contribute a new wireless positioning person re-identification dataset WP-ReID to evaluate our approach. Extensive experiments demonstrate the effectiveness of the proposed method. Benefiting from the collaboration of RCPM and UMM-ReID, the proposed framework MCPF achieves a significant performance improvement over existing methods.
computer science, information systems,telecommunications, software engineering
What problem does this paper attempt to address?