Recognizing human actions using NWFE-based histogram vectors

LinCheng-Hsien,HsuFu-Song,LinWei-Yang

2010-01-01

EURASIP Journal on Advances in Signal Processing

Abstract:This study presents a novel systemfor human action recognition. Two research issues, namely, motion representation and subspace learning, are addressed. In order to have a rich motion descriptor, w...

What problem does this paper attempt to address?

Human Action Recognition Using Deep Learning Methods.

Zeqi Yu,Wei Qi Yan

DOI: https://doi.org/10.1109/ivcnz51579.2020.9290594

2020-01-01

Abstract:The goal of human action recognition is to identify and understand the actions of people in videos and export corresponding tags. In addition to spatial correlation existing in 2D images, actions in a video also own the attributes in temporal domain. Due to the complexity of human actions, e.g., the changes of perspectives, background noises, and others will affect the recognition. In order to solve these thorny problems, three algorithms are designed and implemented in this paper. Based on convolutional neural networks (CNN), Two-Stream CNN, CNN+LSTM, and 3D CNN are harnessed to identify human actions in videos. Each algorithm is explicated and analyzed on details. HMDB-51 dataset is applied to test these algorithms and gain the best results. Experimental results showcase that the three methods have effectively identified human actions given a video, the best algorithm thus is selected.
Student Action Recognition Based on Multiple Features

Fen Lei,Yantao Wei,Jiamin Hu,Huang Yao,Wei Deng,Ying Lu

DOI: https://doi.org/10.1109/ithings/greencom/cpscom/smartdata.2019.00091

2019-01-01

Abstract:Student action analysis plays an important role in learning and teaching. However, manual observation does not meet the needs of big educational data analysis. So it is necessary for us to design an automated way to identify student actions. In this paper, a student action recognition method based on multiple features, which consists of Local Log-Euclidean Multivariate Gaussian (L2EMG) and Scale-Invariant Feature Transform (SIFT), is proposed. L2EMG can model low and high order statistics of local images. SIFT descriptor has strong expansibility, which can be combined with L2EMG algorithm to better describe the features of the image. The experimental results based on self-built dataset have demonstrated that the proposed method has better performance for student action recognition.
Recognizing actions using depth motion maps-based histograms of oriented gradients

Xiaodong Yang,Chenyang Zhang,YingLi Tian

DOI: https://doi.org/10.1145/2393347.2396382

2012-01-01

Abstract:In this paper, we propose an effective method to recognize human actions from sequences of depth maps, which provide additional body shape and motion information for action recognition. In our approach, we project depth maps onto three orthogonal planes and accumulate global activities through entire video sequences to generate the Depth Motion Maps (DMM). Histograms of Oriented Gradients (HOG) are then computed from DMM as the representation of an action video. The recognition results on Microsoft Research (MSR) Action3D dataset show that our approach significantly outperforms the state-of-the-art methods, although our representation is much more compact. In addition, we investigate how many frames are required in our framework to recognize actions on the MSR Action3D dataset. We observe that a short sub-sequence of 30-35 frames is sufficient to achieve comparable results to that operating on entire video sequences.
Subspace Analysis Methods Plus Motion History Image for Human Action Recognition

Chunhua Du,Qiang Wu,Jie Yang,Xiangjian He,Yan Chen

DOI: https://doi.org/10.1109/dicta.2008.20

2008-01-01

Abstract:This paper proposes a new human action recognition method which deals with recognition task in a quite different way when compared with traditional methods which use sequence matching scheme. Our method compresses a sequence of an action into a Motion History Image (MHI) on which low-dimensional features are extracted using subspace analysis methods. Unlike other methods which use a sequence consisting of several frames for recognition, our method uses only a MHI per action sequence for recognition. Obviously, our method avoids the complexity as well as the large computation in sequence matching based methods. Encouraging experimental results on a widely used database demonstrate the effectiveness of the proposed method.
Human Action Recognition Based on DMMs, HOGs and Contourlet Transform

Mohammad Farhad Bulbul,Yunsheng Jiang,Jinwen Ma

DOI: https://doi.org/10.1109/BigMM.2015.82

2015-01-01

Abstract:This paper proposes a framework for recognizing human actions from depth video sequences by designing a novel feature descriptor based on Depth Motion Maps (DMMs), Contour let Transform (CT) and Histogram of Oriented Gradients (HOGs). First, CT is implemented on the generated DMMs of a depth video sequence and then HOGs are computed for each contour let sub-band. Finally, the concatenation of these HOG features is used as a feature descriptor for the depth video sequence. With this new feature descriptor, the l2-regularized collaborative representation classifier is utilized to recognize human actions. The experimental results on Microsoft Research Action3D dataset demonstrate that our proposed method can achieve the state-of-the-art performance for human activity recognition due to the precise feature extraction of contour let transform on the DMMs.
Human Action Recognition Based on Three-Stream Network with Frame Sequence Features

Ruifeng Huang,Chong Chen,Rui Cheng,Ying Zhang,Jiabing Zhu

DOI: https://doi.org/10.1109/icivc55077.2022.9887162

2022-01-01

Abstract:In the field of human action recognition (HAR), two-stream models have been widely employed. In recent years, traditional two-stream network models have disregarded the interframe sequence characteristics of video, resulting in a decrease in model robustness when local sequence information and long-term motion information interact. In light of this, a novel three-stream neural network is proposed by combining the long-term and short-term characteristics of a frame sequence with spatio-temporal information. Initially, the optical flow sequence image frames and RGB image frames in the video are extracted, the optical flow motion information and image space information in the video is obtained, the corresponding time network and space network are entered, and the spatial information is entered into the sequence feature processing network; the three networks are then pretrained. At the conclusion of training, the operation of feature extraction is executed, the features are incorporated with the parallel fusion algorithm by adding weights, and the behavior categories are classified using Multi-Layer Perception. Experimental results on the UCF11, UCF50, and HMDB51 datasets demonstrate that our model effectively integrates the spatial-temporal and frame-sequence information of human actions, resulting in a significant improvement in recognition accuracy. Its classification accuracy on the three datasets was 99.17%, 97.40%, and 96.88%, respectively, significantly enhancing the generalization capability and validity of conventional two-stream or three-stream models.
Full Body Tracking-Based Human Action Recognition

Gu Junxia,Ding Xiaoqing,Wang Shengjin,Wu Youshou

DOI: https://doi.org/10.1109/icpr.2008.4761198

2008-01-01

Abstract:In this paper, we present a novel method for human action recognition with the combined global movement feature and local configuration feature. The human action is represented as a sequence of joints in the 4D spatio-temporal space, and modeled by two HMMs, a conventional HMM for global movement feature and an exemplar-based HMM for configuration feature. Firstly, an adaptive particle filter is adopted to track the marker-less actor's 3D joints. Then, the combined features are extracted from the full body tracking results. Finally, the actions are classified by fusing two HMMs. The effectiveness of the proposed algorithm is demonstrated with experiments on 7 actions by 12 actors. The results prove robustness of the proposed method with respect to viewpoints and actors.
Encoding Learning Network Combined with Feature Similarity Constraints for Human Action Recognition

Chao Wu,Yakun Gao,Guang Li,Chunfeng Shi

DOI: https://doi.org/10.1007/s11042-023-17424-0

IF: 2.577

2023-01-01

Multimedia Tools and Applications

Abstract:Extreme learning machine (ELM) is a fast and efficient classifier. Due to the inability to process descriptor-level features extracted from video sequences, the networks based on ELM cannot be directly used to recognize human actions. Encoding learning network (ELN) is proposed to solve this problem. The network is composed of feature encoding module and double similarity-constrained extreme learning machine (DS-ELM). In feature encoding module, the sparse mapping weight matrix is combined with pyramid pooling to generate representation-level features. DS-ELM is used to classify generated features. In order to utilize the similarity information between the features of each layer, different weight matrices in ELN are separately trained to improve the recognition ability. In the training of sparse mapping weight matrix, the auto-encoded dictionary and similarity constrained linear coding (SCLC) method are proposed to encode the desired output. The sparse mapping weight matrix is trained by using partial descriptor features and corresponding desired outputs. In the training of the classification weights, the ELM objective function is updated by similarity relationship between hidden layer features to derive the training formula of DS-ELM, which improves the classification performance while avoiding iterative training. To verify the feasibility of the ELN, experiments are conducted on Olympic Sports, UCF11, Hollywood2, UCF101, and Self-collection databases. Experimental results show that the proposed ELN is able to directly process descriptor features. And, the similarity information between the features of each layer can be further utilized by ELN to obtain excellent recognition performance compared with other improved methods based on ELM.
Human action recognition in videos using hybrid motion features

Si Liu,Jing Liu,Tianzhu Zhang,Hanqing Lu

DOI: https://doi.org/10.1007/978-3-642-11301-7_42

2010-01-01

Abstract:In this paper, we present hybrid motion features to promote action recognition in videos. The features are composed of two complementary components from different views of motion information. On one hand, the period feature is extracted to capture global motion in time-domain. On the other hand, the enhanced histograms of motion words (EHOM) are proposed to describe local motion information. Each word is represented by optical flow of a frame and the correlations between words are encoded into the transition matrix of a Markov process, and then its stationary distribution is extracted as the final EHOM. Compared to traditional Bags of Words representation, EHOM preserves not only relationships between words but also temporary information in videos to some extent. We show that by integrating local and global features, we get improved recognition rates on a variety of standard datasets.
Human activity recognition method combined with region HOF and dictionary learning

Jian Wang,Chunyu Li,Yuexin Li

DOI: https://doi.org/10.3969/j.issn.1001-3695.2017.09.066

2017-01-01

Abstract:As a secondary care for the elderly,this paper proposed a new human activity recognition method based on video.First,it used AMBER method for detecting the moving objects in video,and located the activity region of interest roughly.Then,it extracted optical flow of each pixel in the activity region of interest,and built normalized histograms of oriented optical flow (HOF),which was used to describe human activities.Finally,it used online dictionary learning method for training and testing,to find the best dictionaries and sparse matrices during training phase,and classified different features according to sparsity during testing phase.The results of experiment on the international human activity dataset show that,recognize human activity by using the new method can achieve high recognition rate,and low category confusion rate between different human activities.
Lf-Eme: Local Features with Elastic Manifold Embedding for Human Action Recognition

Xiaoyu Deng,Xiao Liu,Mingli Song,Jun Cheng,Jiajun Bu,Chun Chen

DOI: https://doi.org/10.1016/j.neucom.2012.06.011

IF: 6

2013-01-01

Neurocomputing

Abstract:Human action recognition has been an active topic in computer vision. Currently, most of the approaches to this problem can be categorized into two classes. One is based on local features, and the other is based on global features. Meanwhile, manifold learning has become successful in many problems in computer vision, but because of the high variability of human body, the application of manifold learning to human action recognition is limited. We propose a framework based on Elastic Manifold Embedding (EME), a new sparse manifold learning algorithm, together with local interest point features to handle human action recognition. The result of the new framework is very promising in comparison with state-of-the-art methods.
Machine Vision-Based Human Action Recognition Using Spatio-Temporal Motion Features (STMF) with Difference Intensity Distance Group Pattern (DIDGP)

Jawaharlalnehru Arunnehru,Sambandham Thalapathiraj,Ravikumar Dhanasekar,Loganathan Vijayaraja,Raju Kannadasan,Arfat Ahmad Khan,Mohd Anul Haq,Mohammed Alshehri,Ismail Keshta,Mohamed Ibrahim Alwanin

DOI: https://doi.org/10.3390/electronics11152363

IF: 2.9

2022-07-29

Electronics

Abstract:In recent years, human action recognition is modeled as a spatial-temporal video volume. Such aspects have recently expanded greatly due to their explosively evolving real-world uses, such as visual surveillance, autonomous driving, and entertainment. Specifically, the spatio-temporal interest points (STIPs) approach has been widely and efficiently used in action representation for recognition. In this work, a novel approach based on the STIPs is proposed for action descriptors i.e., Two Dimensional-Difference Intensity Distance Group Pattern (2D-DIDGP) and Three Dimensional-Difference Intensity Distance Group Pattern (3D-DIDGP) for representing and recognizing the human actions in video sequences. Initially, this approach captures the local motion in a video that is invariant to size and shape changes. This approach extends further to build unique and discriminative feature description methods to enhance the action recognition rate. The transformation methods, such as DCT (Discrete cosine transform), DWT (Discrete wavelet transforms), and hybrid DWT+DCT, are utilized. The proposed approach is validated on the UT-Interaction dataset that has been extensively studied by past researchers. Then, the classification methods, such as Support Vector Machines (SVM) and Random Forest (RF) classifiers, are exploited. From the observed results, it is perceived that the proposed descriptors especially the DIDGP based descriptor yield promising results on action recognition. Notably, the 3D-DIDGP outperforms the state-of-the-art algorithm predominantly.

engineering, electrical & electronic,computer science, information systems,physics, applied
Recognition of Human Activities Using Depth Maps and the Viewpoint Feature Histogram Descriptor

Kamil Sidor,Marian Wysocki

DOI: https://doi.org/10.3390/s20102940

2020-05-22

Abstract:In this paper we propose a way of using depth maps transformed into 3D point clouds to classify human activities. The activities are described as time sequences of feature vectors based on the Viewpoint Feature Histogram descriptor (VFH) computed using the Point Cloud Library. Recognition is performed by two types of classifiers: (i) k-NN nearest neighbors' classifier with Dynamic Time Warping measure, (ii) bidirectional long short-term memory (BiLSTM) deep learning networks. Reduction of classification time for the k-NN by introducing a two tier model and improvement of BiLSTM-based classification via transfer learning and combining multiple networks by fuzzy integral are discussed. Our classification results obtained on two representative datasets: University of Texas at Dallas Multimodal Human Action Dataset and Mining Software Repositories Action 3D Dataset are comparable or better than the current state of the art.
Human Action Recognition in Unconstrained Videos by Explicit Motion Modeling

Yu-Gang Jiang,Qi Dai,Wei Liu,Xiangyang Xue,Chong-Wah Ngo

DOI: https://doi.org/10.1109/tip.2015.2456412

IF: 10.6

2015-01-01

IEEE Transactions on Image Processing

Abstract:Human action recognition in unconstrained videos is a challenging problem with many applications. Most state-of-the-art approaches adopted the well-known bag-of-features representations, generated based on isolated local patches or patch trajectories, where motion patterns, such as object-object and object-background relationships are mostly discarded. In this paper, we propose a simple representation aiming at modeling these motion relationships. We adopt global and local reference points to explicitly characterize motion information, so that the final representation is more robust to camera movements, which widely exist in unconstrained videos. Our approach operates on the top of visual codewords generated on dense local patch trajectories, and therefore, does not require foreground-background separation, which is normally a critical and difficult step in modeling object relationships. Through an extensive set of experimental evaluations, we show that the proposed representation produces a very competitive performance on several challenging benchmark data sets. Further combining it with the standard bag-of-features or Fisher vector representations can lead to substantial improvements.
Human Action Recognition Using Multi-Velocity STIPs and Motion Energy Orientation Histogram.

Chuanzhen Li,Bailiang Su,Jingling Wang,Hui Wang,Qin Zhang

DOI: https://doi.org/10.6688/jise.2014.30.2.2

2014-01-01

Journal of information science and engineering

Abstract:Local image features in space-time or spatio-temporal interest points provide compact and abstract representations of patterns in a video sequence. In this paper, we present a novel human action recognition method based on multi-velocity spatio-temporal interest points (MVSTIPs) and a novel local descriptor called motion energy (ME) orientation histogram (MEOH). The MVSTIP detection includes three steps: first, filtering video frames with multi-direction ME filters at different speeds to detect significant changes at the pixel level; thereafter, a surround suppression model is employed to rectify the ME deviation caused by the camera motion and complicated backgrounds (e.g., dynamic texture); finally, MVSTIPs are obtained with local maximum filters at multispeeds. After detection, we develop MEOH descriptor to capture the motion features in local regions around interest points. The performance of the proposed method is evaluated on KTH, Weizmann, and UCF sports human action datasets. Results show that our method is robust to both simple and complex backgrounds and the method is superior to other methods that are based on local features.
View-invariant action recognition based on local linear dynamical system

Changhong Liu,Yong Chen,Yang Yang

DOI: https://doi.org/10.1109/CISP.2010.5646374

2010-01-01

Abstract:To address recognition of human actions under view changes, this paper proposes a view-invariant human action recognition approach based on local linear dynamical system and sparse coding. We utilize the bag-of-words (BoW) approach, local patches are modeled as linear dynamical systems and the model parameters are used as the descriptors of local patches. The model parameters capture the dynamics in human actions which is insensitive to view changes. The sparse coding algorithm is then applied to learn discriminative codebook and to avoid the initialization problem in the k-means algorithm. The proposed approach is tested on the IXMAS dataset. The experimental results demonstrate that this approach can recognize the view-invariant actions, obtain high recognition rates, and achieve comparable results in cross-views action recognition.
Real-Time Human Action Recognition System Using Depth Map Sequences

Li,Gui Jin Wang,Xing Gang Lin,Guang Cheng,Li He

DOI: https://doi.org/10.4028/www.scientific.net/amr.760-762.1647

2013-01-01

Advanced Materials Research

Abstract:This work presents a real-time human action recognition system that uses depth map sequence as input. The system contains the segmentation of human, the action modeling based on 3D shape context and the action graph algorithm. We effectively solve the problem of segmenting human from complex and cluttered scenes by combing a novel quadtree split-and-merge method and the codebook background modeling algorithm. We aims at recognizing actions that are used in games and interactions, especially complex actions that contain foot motion and body heave. By expanding the shape context descriptor into 3D space, we obtain translation and scale invariant features and get rid of normalization error, which is a common problem of real-life applications. Experiments in various scenarios demonstrate the high speed and excellent performance of our procedure.
Human action recognition using Adaptive Hierarchical Depth Motion Maps and Gabor filter

Hong Liu,Qinqin He,Mengyuan Liu

DOI: https://doi.org/10.1109/ICASSP.2017.7952393

2017-01-01

Abstract:Depth motion maps (DMMs) have shown effectiveness in human action recognition, however, they lose the temporal information and suffer from intra-class variations caused by action speed variations. To address these challenges, we propose a novel method for human action recognition. Firstly, Adaptive Hierarchical Depth Motion Maps (AH-DMMs) are calculated over temporal hierarchical windows of video sequences to capture the temporal information. Moreover, adaptive windows and steps are employed to ensure that AH-DMMs are robust to motion speed variations. Then, Gabor filter is adopted to encode the texture information of AH-DMMs, generating compact and discriminative action representations. Finally, the representations serve as the input of collaborative representation classifier (CRC). Experimental results on public benchmark MSRAction3D dataset and DHA dataset demonstrate the superiority of the proposed method over the state-of-the-art depth-based action recognition approaches.
Multi-Temporal Depth Motion Maps-Based Local Binary Patterns for 3-D Human Action Recognition

Chen,Mengyuan Liu,Hong Liu,Baochang Zhang,Jungong Han,Nasser Kehtarnavaz

DOI: https://doi.org/10.1109/access.2017.2759058

IF: 3.9

2017-01-01

IEEE Access

Abstract:This paper presents a local spatio-temporal descriptor for action recognistion from depth video sequences, which is capable of distinguishing similar actions as well as coping with different speeds of actions. This descriptor is based on three processing stages. In the first stage, the shape and motion cues are captured from a weighted depth sequence by temporally overlapped depth segments, leading to three improved depth motion maps (DMMs) compared with the previously introduced DMMs. In the second stage, the improved DMMs are partitioned into dense patches, from which the local binary patterns histogram features are extracted to characterize local rotation invariant texture information. In the final stage, a Fisher kernel is used for generating a compact feature representation, which is then combined with a kernel-based extreme learning machine classifier. The developed solution is applied to five public domain data sets and is extensively evaluated. The results obtained demonstrate the effectiveness of this solution as compared with the existing approaches.
Human Action Recognition Based on HOIRM Feature Fusion and AP Clustering BOW.

Ruo-Hong Huan,Chao-Jie Xie,Feng Guo,Kai-Kai Chi,Ke-Ji Mao,Ying-Long Li,Yun Pan

DOI: https://doi.org/10.1371/journal.pone.0219910

IF: 3.7

2019-01-01

PLoS ONE

Abstract:In this paper, we propose a human action recognition method using HOIRM (histogram of oriented interest region motion) feature fusion and a BOW (bag of words) model based on AP (affinity propagation) clustering. First, a HOIRM feature extraction method based on spatiotemporal interest points ROI is proposed. HOIRM can be regarded as a middle-level feature between local and global features. Then, HOIRM is fused with 3D HOG and 3D HOF local features using a cumulative histogram. The method further improves the robustness of local features to camera view angle and distance variations in complex scenes, which in turn improves the correct rate of action recognition. Finally, a BOW model based on AP clustering is proposed and applied to action classification. It obtains the appropriate visual dictionary capacity and achieves better clustering effect for the joint description of a variety of features. The experimental results demonstrate that by using the fused features with the proposed BOW model, the average recognition rate is 95.75% in the KTH database, and 88.25% in the UCF database, which are both higher than those by using only 3D HOG+3D HOF or HOIRM features. Moreover, the average recognition rate achieved by the proposed method in the two databases is higher than that obtained by other methods.

Recognizing human actions using NWFE-based histogram vectors

Human Action Recognition Using Deep Learning Methods.

Student Action Recognition Based on Multiple Features

Recognizing actions using depth motion maps-based histograms of oriented gradients

Subspace Analysis Methods Plus Motion History Image for Human Action Recognition

Human Action Recognition Based on DMMs, HOGs and Contourlet Transform

Human Action Recognition Based on Three-Stream Network with Frame Sequence Features

Full Body Tracking-Based Human Action Recognition

Encoding Learning Network Combined with Feature Similarity Constraints for Human Action Recognition

Human action recognition in videos using hybrid motion features

Human activity recognition method combined with region HOF and dictionary learning

Lf-Eme: Local Features with Elastic Manifold Embedding for Human Action Recognition

Machine Vision-Based Human Action Recognition Using Spatio-Temporal Motion Features (STMF) with Difference Intensity Distance Group Pattern (DIDGP)

Recognition of Human Activities Using Depth Maps and the Viewpoint Feature Histogram Descriptor

Human Action Recognition in Unconstrained Videos by Explicit Motion Modeling

Human Action Recognition Using Multi-Velocity STIPs and Motion Energy Orientation Histogram.

View-invariant action recognition based on local linear dynamical system

Real-Time Human Action Recognition System Using Depth Map Sequences

Human action recognition using Adaptive Hierarchical Depth Motion Maps and Gabor filter

Multi-Temporal Depth Motion Maps-Based Local Binary Patterns for 3-D Human Action Recognition

Human Action Recognition Based on HOIRM Feature Fusion and AP Clustering BOW.