Abstract:As image processing techniques and devices advance, the real-time applications of computer vision such as human action/interaction recognition and video content analysis become more attractive. However, the methods proposed in the state-of-the-art studies are still far from representing real-time and all-inclusive classifiers because of the image and video analysis complexity. This work presents a new approach based on key-poses of frame silhouettes for human interaction recognition. We use an inner-distance-based shape descriptor which gives a perfect description of the shape due to its ability to collect data from the whole shape. The core idea is to develop a two-step classifier based on a sequential pattern mining classifier. So, we extract the Bilateral Silhouette shape for the persons and describe it based on the inner-distance feature to compare each frame with a pre-defined dictionary of key-poses. The classification process is performed in frame and sequence layers. Accurate and efficient, the sequential pattern mining approach provides an appealing solution to the problem of sequence classification, giving comparable or even better results than standard classifiers. We evaluated the recognition performance of the system using video sequences of SBU human interaction dataset and the UT-interaction dataset as two well-known interaction datasets and the results are considered acceptable (95.25% in SBU and 90.5% in UT databases, respectively), outperforming most state-of-the-art results. These recognition rates are calculated after we have tested different parameters which can affect the results. Both datasets include multiple interaction classes performed by different actors, which helps us develop an all-inclusive method based on the datasets. The proposed method can be optimized to be used in some real world applications such as abnormal activity recognition in crowded places, auxiliary surveillance system, human-computer interaction, etc.

Multimodal Multi-User Surface Recognition with the Kernel Two-Sample Test

Multiresolution Match Kernels for Gesture Video Classification

Multimodal behavior analysis in computer-enabled laboratories using nonverbal cues

SurfaceAug: Closing the Gap in Multimodal Ground Truth Sampling

Two-person interaction recognition using a two-step sequential pattern classification

Research on Image Recognition Technology Based on Multimodal Deep Learning

Robust Surface Recognition with the Maximum Mean Discrepancy: Degrading Haptic-Auditory Signals through Bandwidth and Noise

Kernel-Free Universum Quadratic Surface Twin Support Vector Machines for Imbalanced Data

Multimodal data fusion using signal/image processing methods for multi-class machine learning

Multimodal Surface Sensing Based on Hybrid Flexible Triboelectric and Piezoresistive Sensor.

Surface Material Recognition Using Active Multi-modal Extreme Learning Machine.

Deep Multitask Architecture for Integrated 2D and 3D Human Sensing

Human-Exploratory-Procedure-Based Hybrid Measurement Fusion for Material Recognition

Surface Material Retrieval Using Weakly Paired Cross-Modal Learning

Cross-Modal Surface Material Retrieval Using Discriminant Adversarial Learning

A Generic Multi-modal Dynamic Gesture Recognition System using Machine Learning

Multiple Riemannian Manifold-valued Descriptors based Image Set Classification with Multi-Kernel Metric Learning

Audio-visual cross-modality knowledge transfer for machine learning-based in-situ monitoring in laser additive manufacturing

Multimodal Material Classification for Robots using Spectroscopy and High Resolution Texture Imaging

Supervised kernel-based multi-modal Bhattacharya distance learning for imbalanced data classification