PRISM: A System of Evaluating Trustworthiness of IoT by Multi-Modal Surveillance

Can Su,Xinlei Xue,Wei Yan,Kaigui Bian
DOI: https://doi.org/10.1109/iccsn57992.2023.10297341
2023-01-01
Abstract:In this paper, we propose PRISM, a framework to evaluate trustworthiness of IoT system by multi-modal surveillance. Traditional surveillance methods can identify target people's trajectory and check objects' status in the IoT system, which mainly leverage video cameras to observe target people or objects by their appearance features through vision data. Nevertheless, such methods can easily be disturbed and fail in real scenes, such as occlusion, different illuminating conditions, or similar appearances of multiple targets. On the other hand, most people carry Wi-Fi enabled mobile phones with them, whose wireless signals can be sensed by Wi-Fi access points for positioning the person. The wireless signals are robust to vision disturbances, but are inevitable to introduce localization errors, nor can they provide the actual figures of the person. In this work, we combine videos and wireless signals and compliment one another for a novel multi-modal surveillance. We match videos and wireless signals of the same target and fuse for more representative features. Our method not only outperforms traditional surveillance method by over 45% in accuracy, but also improves overall matching rankings for all correct candidates.
What problem does this paper attempt to address?