Multi-Channel Time-Series Person and Soft-Biometric Identification

Nilah Ravi Nair,Fernando Moya Rueda,Christopher Reining,Gernot A. Fink

2023-04-04

Abstract:Multi-channel time-series datasets are popular in the context of human activity recognition (HAR). On-body device (OBD) recordings of human movements are often preferred for HAR applications not only for their reliability but as an approach for identity protection, e.g., in industrial settings. Contradictory, the gait activity is a biometric, as the cyclic movement is distinctive and collectable. In addition, the gait cycle has proven to contain soft-biometric information of human groups, such as age and height. Though general human movements have not been considered a biometric, they might contain identity information. This work investigates person and soft-biometrics identification from OBD recordings of humans performing different activities using deep architectures. Furthermore, we propose the use of attribute representation for soft-biometric identification. We evaluate the method on four datasets of multi-channel time-series HAR, measuring the performance of a person and soft-biometrics identification and its relation concerning performed activities. We find that person identification is not limited to gait activity. The impact of activities on the identification performance was found to be training and dataset specific. Soft-biometric based attribute representation shows promising results and emphasis the necessity of larger datasets.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to use multi - channel time - series data for personal identity and soft biometric identification from human activity recognition. Specifically, the research focuses on the following aspects:

1. **Personal Identity Recognition**: The paper explores the possibility of using multi - channel time - series data recorded by on - body devices (OBDs) to recognize personal identities. These devices usually include smart phones or smart watches and can record human motion data.

2. **Soft Biometric Identification**: In addition to personal identity recognition, the paper also studies how to extract information on soft biometrics (such as age, gender, height) from the data recorded by OBDs and use it as an attribute representation for identification.

3. **The Influence of Activities on Recognition Performance**: The paper analyzes the influence of different activity types on the performance of personal identity and soft biometric identification. The research finds that some activities (such as walking) can provide more useful identification information than other activities (such as sitting quietly).

4. **Standardization of Datasets and Methods**: To ensure the reproducibility and standardization of experiments, the paper conducts experiments on four publicly available multi - channel time - series datasets, namely LARa, OPPORTUNITY, PAMAP2 and Order Picking. The research uses deep neural networks (DNNs), especially convolutional neural networks (CNNs) and recurrent neural networks (RNNs) to perform identification tasks.

5. **The Application of Explainable Artificial Intelligence (XAI)**: To verify the validity of the identification results, the paper also attempts to use explainable artificial intelligence methods (such as Layer - Wise Relevance Propagation, LRP) to analyze which features contribute the most to personal identity recognition.

Through these studies, the paper aims to promote the development of efficient identity protection methods and provide impetus for the creation of large - scale labeled datasets to facilitate further research in related fields.