Abstract:As image processing techniques and devices advance, the real-time applications of computer vision such as human action/interaction recognition and video content analysis become more attractive. However, the methods proposed in the state-of-the-art studies are still far from representing real-time and all-inclusive classifiers because of the image and video analysis complexity. This work presents a new approach based on key-poses of frame silhouettes for human interaction recognition. We use an inner-distance-based shape descriptor which gives a perfect description of the shape due to its ability to collect data from the whole shape. The core idea is to develop a two-step classifier based on a sequential pattern mining classifier. So, we extract the Bilateral Silhouette shape for the persons and describe it based on the inner-distance feature to compare each frame with a pre-defined dictionary of key-poses. The classification process is performed in frame and sequence layers. Accurate and efficient, the sequential pattern mining approach provides an appealing solution to the problem of sequence classification, giving comparable or even better results than standard classifiers. We evaluated the recognition performance of the system using video sequences of SBU human interaction dataset and the UT-interaction dataset as two well-known interaction datasets and the results are considered acceptable (95.25% in SBU and 90.5% in UT databases, respectively), outperforming most state-of-the-art results. These recognition rates are calculated after we have tested different parameters which can affect the results. Both datasets include multiple interaction classes performed by different actors, which helps us develop an all-inclusive method based on the datasets. The proposed method can be optimized to be used in some real world applications such as abnormal activity recognition in crowded places, auxiliary surveillance system, human-computer interaction, etc.

ISL Person Identification Systems in the CLEAR 2007 Evaluations

Rapid: A Multimodal and Device-free Approach Using Noise Estimation for Robust Person Identification

Audio-visual multi-person tracking and identification for smart environments

Multimodal person authentication using speech, face and visual speech

Intelligent Sensor Information System For Public Transport - To Safely Go

A PC-Based System for Automated Iris Recognition under Open Environment

Person Identification with Visual Summary for a Safe Access to a Smart Home

A Vision System for Multi-View Face Recognition

Multi-camera Handoff for Person Re-Identification

Two-person interaction recognition using a two-step sequential pattern classification

A Multi-Biometrics for Twins Identification Based Speech and Ear

Prediction based person recognition using face and speech (multi modal) for improved performance

Comparative Analysis of Modality Fusion Approaches for Audio-Visual Person Identification and Verification

Enhancing societal security: a multimodal deep learning approach for a public person identification and tracking system

Cooperative Separation of Modality Shared-Specific Features for Visible-Infrared Person Re-Identification

[On the posoperative period in cataract extraction].

Technology evaluations on the TH-FACE recognition system

Tsinghua Face Detection and Tracking for CLEAR 2007 Evaluation.

Amazigh audiovisual speech recognition system design

A Robust Single-sensor Face and Iris Biometric Identification System Based on Multimodal Feature Extraction Network

Three-Stage Speaker Verification Architecture in Emotional Talking Environments