Abstract:The human gait pattern is an emerging biometric trait for user identification of smart devices. However, one of the challenges in this biometric domain is the gait pattern change caused by footwear, especially if the users are wearing high heels (HH). Wearing HH puts extra stress and pressure on various parts of the human body and it alters the wearer’s common gait pattern, which may cause difficulties in gait recognition. In this paper, we propose the Sensing-HH, a deep hybrid attention model for recognizing the subject’s shoes, flat or different types of HH, using smartphone’s motion sensors. In this model, two streams of convolutional and bidirectional long short-term memory (LSTM) networks are designed as the backbone, which extract the hierarchical spatial and temporal representations of accelerometer and gyroscope individually. We also introduce a spatio attention mechanism into the stacked convolutional layers to scan the crucial structure of the data. This mechanism enables the hybrid neural networks to capture extra information from the signal and thus it is able to significantly improve the discriminative power of the classifier for the footwear recognition task. To evaluate Sensing-HH, we built a dataset with 35 young females, each of whom walked for 4 min wearing shoes with varied heights of the heels. We conducted extensive experiments and the results demonstrated that the Sensing-HH outperformed the baseline models on leave-one-subject-out cross-validation (LOSO-CV). The Sensing-HH achieved the best Fm score, which was 0.827 when the smartphone was attached to the waist. This outperformed all the baseline methods at least by more than 14%. Meanwhile, the F1 Score of the Ultra HH was as high as 0.91. The results suggest the proposed model has made the footwear recognition more efficient and automated. We hope the findings from this study paves the way for a more sophisticated application using data from motion sensors, as well as lead to a path to a more robust biometric system based on gait pattern.

Learning Wear Patterns on Footwear Outsoles Using Convolutional Neural Networks

Deep learning model for human-intuitive shoeprint reconstruction

Sexing Caucasian 2D footprints using convolutional neural networks

Classification of footwear outsole patterns using Fourier transform and local interest points

Deep Multilabel CNN for Forensic Footwear Impression Descriptor Identification

Improving and Evaluating Machine Learning Methods for Forensic Shoeprint Matching

Creating a Forensic Database of Shoeprints from Online Shoe Tread Photos

A novel method to predict slip resistance of winter footwear using a convolutional neural network

Overlapping Shoeprint Detection by Edge Detection and Deep Learning

Deep Learning Analysis and Age Prediction from Shoeprints

Classification of 3D shoe prints using the PointNet architecture: proof of concept investigation of binary classification of nike and adidas outsoles

Novel Soft Smart Shoes for Motion Intent Learning of Lower Limbs Using LSTM with a Convolutional Autoencoder

CriSp: Leveraging Tread Depth Maps for Enhanced Crime-Scene Shoeprint Matching

Closing the Wearable Gap: Foot-ankle kinematic modeling via deep learning models based on a smart sock wearable

Estimating the slip resistant quality of winter footwear using Artificial Intelligence

LaserShoes: Low-Cost Ground Surface Detection Using Laser Speckle Imaging.

Virtual Footwear Try-on in Augmented Reality using Deep Learning Models

Image Classifier for an Online Footwear Marketplace to Distinguish between Counterfeit and Real Sneakers for Resale

An Ensemble Deep Neural Network for Footprint Image Retrieval Based on Transfer Learning

Forensic shoe-print identification: a brief survey

Sensing-HH: A Deep Hybrid Attention Model for Footwear Recognition