Recognition of Humans and Their Activities Using Video

Rama Chellappa,Amit K. Roy-Chowdhury,S. Kevin Zhou

DOI: https://doi.org/10.1007/978-3-031-02236-4

2005-01-01

Synthesis Lectures on Image, Video, and Multimedia Processing

Abstract:The recognition of humans and their activities from video sequences is currently a very active area of research because of its applications in video surveillance, design of realistic entertainment sys

What problem does this paper attempt to address?

Human Action Recognition

Shraddha Marotkar -,A.B. Kharate -

DOI: https://doi.org/10.36948/ijfmr.2023.v05i02.1743

2023-03-03

International Journal For Multidisciplinary Research

Abstract:human-to-human interaction and interpersonal relations Human activity recognition plays a significant role. Because to identity of a person, their personality, and psychological state it provides information, it is difficult to extract. another person's activities is one of the main subjects of study of the scientific areas of computer vision The human ability to recognize and soft computing learning. robotics for human behavior characterization, require a multiple activity recognition system, many applications, including video surveillance systems, human computer interaction, and human activity recognition is an important research direction In video analysis, a large number of papers have been published on human activity recognition in video and image sequences In the past, including methods, systems, and quantitative evaluation of the performance of human activity recognition In this paper, we provide a comprehensive survey of the recent development of the techniques, for the video improve classification, interpretation, and performance. The experimental results show that our method can significantly.
Human Action Recognition Using Deep Learning Methods.

Zeqi Yu,Wei Qi Yan

DOI: https://doi.org/10.1109/ivcnz51579.2020.9290594

2020-01-01

Abstract:The goal of human action recognition is to identify and understand the actions of people in videos and export corresponding tags. In addition to spatial correlation existing in 2D images, actions in a video also own the attributes in temporal domain. Due to the complexity of human actions, e.g., the changes of perspectives, background noises, and others will affect the recognition. In order to solve these thorny problems, three algorithms are designed and implemented in this paper. Based on convolutional neural networks (CNN), Two-Stream CNN, CNN+LSTM, and 3D CNN are harnessed to identify human actions in videos. Each algorithm is explicated and analyzed on details. HMDB-51 dataset is applied to test these algorithms and gain the best results. Experimental results showcase that the three methods have effectively identified human actions given a video, the best algorithm thus is selected.
Human activity recognition for video surveillance

Weiyao Lin,Ming-ting Sun,Radha Poovendran,Zhengyou Zhang

DOI: https://doi.org/10.1109/ISCAS.2008.4542023

2008-01-01

Abstract:This paper presents a novel approach for automatic recognition of human activities from video sequences. We first group features with high correlations into category feature vectors (CFVs). Each activity is then described by a combination of GMMs (Gaussian mixture models) with each GMM representing the distribution of a CFV. We show that this approach offers flexibility to add new events and to deal with the problem of lacking training data for building models for unusual events. For improving the recognition accuracy, a confident-frame-based Recognizing algorithm (CFR) is proposed to recognize the human activity, where the video frames which have high confidence for recognition an activity (confident-frames) are used as a specialized model for classifying the rest of the video frames. Experimental results show the effectiveness of the proposed approach.
A Survey Of Activity Recognition And Understanding The Behavior In Video Survelliance

A. R. Revathi,Dhananjay Kumar

DOI: https://doi.org/10.48550/arXiv.1207.6774

2012-07-29

Computer Vision and Pattern Recognition

Abstract:This paper presents a review of human activity recognition and behaviour understanding in video sequence. The key objective of this paper is to provide a general review on the overall process of a surveillance system used in the current trend. Visual surveillance system is directed on automatic identification of events of interest, especially on tracking and classification of moving objects. The processing step of the video surveillance system includes the following stages: Surrounding model, object representation, object tracking, activity recognition and behaviour understanding. It describes techniques that use to define a general set of activities that are applicable to a wide range of scenes and environments in video sequence.
A human activity recognition framework in videos using segmented human subject focus

Shaurya Gupta,Dinesh Kumar Vishwakarma,Nitin Kumar Puri

DOI: https://doi.org/10.1007/s00371-023-03256-4

IF: 2.835

2024-02-08

The Visual Computer

Abstract:Automating tasks through human activity recognition in video data has become increasingly vital. Deep learning has yielded versatile activity recognition systems applicable in surveillance, healthcare analysis, sports, and human–computer interaction. Despite various proposed video-based activity recognition techniques over the years, the reliance over RGB frames, accompanied by other modalities like joint locations and depth maps, often proves less effective compared to multimodal methods. In response to this challenge, our paper introduces a competitive approach for identifying human activity in video frames. Leveraging a Convolutional Long Short-Term Memory (Conv-LSTM) network and a novel pre-processing step involving a Human Segmentation network, our method accentuates human subjects in each frame using segmentation maps. These highlighted frames undergo further processing through Convolutional Neural Networks (CNNs) to learn feature vectors, without including other modalities with RGB frames directly. The learned features are then subjected to Long Short-Term Memory (LSTM) units for comprehending sequential video data and drawing meaningful inferences. The proposed methodology undergoes rigorous testing on three publicly available datasets—KARD, MSR Daily Activity, and SBU-Interactions. Remarkably, our approach outperforms similar state-of-the-art methods, achieving benchmark accuracy scores exceeding 98% on MSR Daily Activity and 99% on KARD and SBU-Interactions datasets. In essence, our method not only provides a competitive solution for human activity recognition in video frames but also contributes to advancing the field by integrating Conv-LSTM networks and innovative pre-processing techniques. The comprehensive evaluation on multiple datasets underlines the robustness and superior performance of our proposed approach.

computer science, software engineering
Video-understanding framework for automatic behavior recognition

François Brémond,Monique Thonnat,Marcos Zúñiga

DOI: https://doi.org/10.3758/bf03192795

IF: 5.953

2006-08-01

Behavior Research Methods

Abstract:We propose an activity-monitoring framework based on a platform called VSIP, enabling behavior recognition in different environments. To allow end-users to actively participate in the development of a new application, VSIP separates algorithms from a priori knowledge. To describe how VSIP works, we present a full description of a system developed with this platform for recognizing behaviors, involving either isolated individuals, groups of people, or crowds, in the context of visual monitoring of metro scenes, using multiple cameras. In this work, we also illustrate the capability of the framework to easily combine and tune various recognition methods dedicated to the visual analysis of specific situations (e.g., mono-/multiactors’ activities, numerical/symbolic actions, or temporal scenarios). We also present other applications, using this framework, in the context of behavior recognition. VSIP has shown a good performance on human behavior recognition for different problems and configurations, being suitable to fulfill a large variety of requirements.

psychology, experimental, mathematical
Effective approaches in human action recognition

xue li,quan z sheng,chaoyi pang,xin zhao,sen wang

DOI: https://doi.org/10.1109/ICACSIS.2013.6761544

2013-01-01

Abstract:Recognising and understanding the activities performed by people is a fundamental research topic in developing a wi de range of applications that would be societally beneficial. In this article, we present and discuss two research projects on human action recognition based on computer vision techniques. We also report an ongoing research project that focuses on learning human activities through low cost, unobtrusiveradio frequency identification (RFID) technology.
Multi-object Events Recognition from Video Sequences Using Extended Finite State Machine

Chun Yuan,Wei Xu

DOI: https://doi.org/10.1109/cisp.2011.6099975

2011-01-01

Abstract:Representation and recognition of human activities or other events generated by vehicles in restricted settings such as airports, railway station and parking lots is of great importance in automated surveillance systems. The problem is difficult because: 1) at the lower level, detection of primitive actions is inaccurate due to changes in illumination, noise and resolution of video camera. 2) at the higher level, activities or events in such settings are usually originated by multi-object. It is hard to represent and parse the complex activities and events. In this paper, we describe a new approach for representing and recognizing the events in video sequences, and the approach is able to handle complex multi-agent interactions. Our experimental results on a dataset that consisting of videos took by normal camera and videos took by surveillance camera in laboratory.
A recent survey for human activity recoginition based on deep learning approach

Jagwinder Kaur Dhillon,Chandni,Alok Kumar Singh Kushwaha

DOI: https://doi.org/10.1109/iciip.2017.8313715

2017-12-01

Abstract:Human activity recognition is an active research topic in computer vision due to its applicability to wide range of application areas such as smart surveillance, robot learning, human computer interaction, health assessment. Identifying human activities from video sequences constitutes one of the most challenging tasks in the field of computer vision, especially due to the harsh nature of the real-world activity recognition scenarios and high volumes of data that need to be worked upon. The techniques available in the literature for activity recognition are broadly classified into two categories: single layered approaches aimed at recognition of much simpler activities, hierarchical approaches aimed at recognition of complex activities in terms of simpler ones. Deep learning is one of the hierarchical approaches for recognizing human activities capable of achieving outstanding results and outperforming other “non-deep” state-of-the-art methods by effectively utilizing the image structure in reducing the search space of the learning model. This paper aims at capturing a snapshot of current trends in activity recognition with deep learning models. We have also examined the merits, demerits, efficiency of pioneering deep learning models being used for activity recognition.
Recognition of Human Activities Using Svm Multi-Class Classifier

Huimin Qian,Yaobin Mao,Wenbo Xiang,Zhiquan Wang

DOI: https://doi.org/10.1016/j.patrec.2009.09.019

IF: 4.757

2010-01-01

Pattern Recognition Letters

Abstract:Even great efforts have been made for decades, the recognition of human activities is still an unmature technology that attracted plenty of people in computer vision. In this paper, a system framework is presented to recognize multiple kinds of activities from videos by an SVM multi-class classifier with a binary tree architecture. The framework is composed of three functionally cascaded modules: (a) detecting and locating people by non-parameter background subtraction approach, (b) extracting various of features such as local ones from the minimum bounding boxes of human blobs in each frames and a newly defined global one, contour coding of the motion energy image (CCMEI), and (c) recognizing activities of people by SVM multi-class classifier whose structure is determined by a clustering process. The thought of hierarchical classification is introduced and multiple SVMs are aggregated to accomplish the recognition of actions. Each SVM in the multi-class classifier is trained separately to achieve its best classification performance by choosing proper features before they are aggregated. Experimental results both on a home-brewed activity data set and the public Schüldt’s data set show the perfect identification performance and high robustness of the system.
Real Time Human Action Recognition in a Long Video Sequence

Ping Guo,Zhenjiang Miao,Yuan Shen,Heng-Da Cheng

DOI: https://doi.org/10.1109/avss.2010.44

2010-01-01

Abstract:In recent years, most action recognition researches focuson isolated action analysis for short videos, but ignore theissue of continuous action recognition for a long videosequence in real time. This paper proposes a novelapproach for human action recognition in a video sequencewith whatever length, which, unlike previous works,requires no annotations and no pre-temporal-segmentations.Based on the bag of words representation and theprobabilistic Latent Semantic Analysis (pLSA) model, therecognition process goes frame by frame and the decisionupdates from time to time. Experimental results show thatthis approach is effective to recognize both isolated actionsand continuous actions no matter how long a videosequence is. This is very useful for real time applicationslike video surveillance. Besides, we also test our approachfor real time temporal video segmentation and real time keyframe extraction.
Human behavior understanding for video surveillance: Recent advance

Xin Xu,Jinshan Tang,Xiaoming Liu,Xiaolong Zhang

DOI: https://doi.org/10.1109/ICSMC.2010.5641773

2010-01-01

Abstract:With the wide applications of video cameras in surveillance, video analysis technologies have attracted the attention from the researchers in computer vision field. In video analysis, human behavior recognition and understanding is an important research direction. By recognition and understanding the human behaviors, we can predict and recognize the happening of crimes and help to the police or other agencies to react immediately. In the past, large amount of intensive papers have been published on human behavior understanding in videos. Generally speaking, the procedure of human behavior understanding can be divided into the following stages: human segmentation and tracking, and human behavior recognition. In this paper, we provide a comprehensive survey of the recent development of all these stages. We will also discuss the difficulties in behavior understanding and identify possible future directions.
Video-Based Abnormal Human Behavior Recognition - A Review.

Oluwatoyin P. Popoola,Kejun Wang

DOI: https://doi.org/10.1109/tsmcc.2011.2178594

2012-01-01

IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews)

Abstract:Modeling human behaviors and activity patterns for recognition or detection of special event has attracted significant research interest in recent years. Diverse methods that are abound for building intelligent vision systems aimed at scene understanding and making correct semantic inference from the observed dynamics of moving targets. Most applications are in surveillance, video content retrieval, and human-computer interfaces. This paper presents not only an update extending previous related surveys, but also a focus on contextual abnormal human behavior detection especially in video surveillance applications. The main purpose of this survey is to extensively identify existing methods and characterize the literature in a manner that brings key challenges to attention.
Human Interaction Recognition Fusing Multiple Features of Depth Sequences

Jianjun Li,Xia Mao,Lijiang Chen,Lan Wang

DOI: https://doi.org/10.1049/iet-cvi.2017.0025

IF: 1.484

2017-01-01

IET Computer Vision

Abstract:Human interaction recognition has played a major role in building intelligent video surveillance systems. Recently, depth data captured by the emerging RGB‐D sensors began to show its importability in human interaction recognition. This study proposes a novel framework for human interaction recognition using depth information including an algorithm to reconstruct depth sequence with as few key frames as possible. The proposed framework includes two essential modules. First, key frames extraction by sparse constraint, then the fusion multi‐feature, is constructed by using two types of available features and Max‐pooling, respectively. Finally, multiple features are directly sent to the SVM for the recognition of the human activity. This study explores the static and dynamic feature fusion method to improve the recognition performance with contextual relevance of continuous frames. A weight is used to fuse shape and optical flow features, which not only enhance the description capability of human behavioural characteristics in the spatiotemporal domain, but also effectively reduces the adverse impact of certain distortion point of interest for target recognition. Experimental results show that the proposed approach yields considerable performance improvement over the state‐of‐the‐art approaches with respect to accuracy on a public action dataset.
Human Activity Recognition based on Dynamic Spatio-Temporal Relations

Zhenyu Liu,Yaqiang Yao,Yan Liu,Yuening Zhu,Zhenchao Tao,Lei Wang,Yuhong Feng

DOI: https://doi.org/10.48550/arXiv.2006.16132

2020-06-29

Abstract:Human activity, which usually consists of several actions, generally covers interactions among persons and or objects. In particular, human actions involve certain spatial and temporal relationships, are the components of more complicated activity, and evolve dynamically over time. Therefore, the description of a single human action and the modeling of the evolution of successive human actions are two major issues in human activity recognition. In this paper, we develop a method for human activity recognition that tackles these two issues. In the proposed method, an activity is divided into several successive actions represented by spatio temporal patterns, and the evolution of these actions are captured by a sequential model. A refined comprehensive spatio temporal graph is utilized to represent a single action, which is a qualitative representation of a human action incorporating both the spatial and temporal relations of the participant objects. Next, a discrete hidden Markov model is applied to model the evolution of action sequences. Moreover, a fully automatic partition method is proposed to divide a long-term human activity video into several human actions based on variational objects and qualitative spatial relations. Finally, a hierarchical decomposition of the human body is introduced to obtain a discriminative representation for a single action. Experimental results on the Cornell Activity Dataset demonstrate the efficiency and effectiveness of the proposed approach, which will enable long videos of human activity to be better recognized.

Computer Vision and Pattern Recognition,Machine Learning
Human behaviour recognition with mid-level representations for crowd understanding and analysis

Bangyong Sun,Nianzeng Yuan,Shuying Li,Siyuan Wu,Nan Wang

DOI: https://doi.org/10.1049/IPR2.12147

IF: 2.3

2021-02-25

IET Image Processing

Abstract:Crowd understanding and analysis have received increasing attention for couples of decades, and development of human behaviour recognition strongly supports the application of crowd understanding and analysis. Human behaviour recognition usually seeks to automatically analyse ongoing movements and actions in different camera views by using various machine learning methodologies in unknown video clips or image sequences. Compared to other data modalities such as documents and images, processing video data demands much higher computational and storage resources. The idea of using middle level semantic concepts to represent human actions from videos is explored and it is argued that these semantic attributes enable the construction of more descriptive methods for human action recognition. The mid-level attributes, initialized by a cluster processing, are built upon low level features and fully utilize the discrepancies in different action classes, which can capture the importance of each attribute for each action class. In this way, the representation is constructed to be semantically rich and capable of highly discriminative performance even paired with simple linear classiﬁers. The method is veriﬁed on three challenging datasets (KTH, UCF50 and HMDB51), and the experimental results demonstrate that our method achieves better results than the baseline methods on human action recognition.

Computer Science
Human Activity Recognition Based On Video Summarization And Deep Convolutional Neural Network

Arati Kushwaha,Manish Khare,Reddy Mounika Bommisetty,Ashish Khare

DOI: https://doi.org/10.1093/comjnl/bxae028

2024-03-23

The Computer Journal

Abstract:Abstract In this technological era, human activity recognition (HAR) plays a significant role in several applications like surveillance, health services, Internet of Things, etc. Recent advancements in deep learning and video summarization have motivated us to integrate these techniques for HAR. This paper introduces a computationally efficient HAR technique based on a deep learning framework, which works well in realistic and multi-view environments. Deep convolutional neural networks (DCNNs) normally suffer from different constraints, including data size dependencies, computational complexity, overfitting, training challenges and vanishing gradients. Additionally, with the use of advanced mobile vision devices, the demand for computationally efficient HAR algorithms with the requirement of limited computational resources is high. To address these issues, we used integration of DCNN with video summarization using keyframes. The proposed technique offers a solution that enhances performance with efficient resource utilization. For this, first, we designed a lightweight and computationally efficient deep learning architecture based on the concept of identity skip connections (features reusability), which preserves the gradient loss attenuation and can handle the enormous complexity of activity classes. Subsequently, we employed an efficient keyframe extraction technique to minimize redundancy and succinctly encapsulate the entire video content in a lesser number of frames. To evaluate the efficacy of the proposed method, we performed the experimentation on several publicly available datasets. The performance of the proposed method is measured in terms of evaluation parameters Precision, Recall, F-Measure and Classification Accuracy. The experimental results demonstrated the superiority of the presented algorithm over other existing state-of-the-art methods.

computer science, information systems, theory & methods, software engineering, hardware & architecture
Fast and Reliable Human Action Recognition in Video Sequences by Sequential Analysis

Hui Fang,Jeyarajan Thiyagalingam,Nik Bessis,Eran Edirisinghe

DOI: https://doi.org/10.1109/icip.2017.8297028

2017-01-01

ICIP

Abstract:Human action recognition from video sequences is a challenging topic in computer vision research. In recent years, many studies have explored the use of deep learning representations to consistently improve the analysis accuracy. Meanwhile, designing a fast and reliable framework is becoming increasingly important given the exponential growth of video data collected for many purposes (e.g. public security, entertainment, and early medical diagnosis etc.). In order to design a more efficient automatic human action annotation method, the sequential probability ratio test, one of the classical statistical sampling scheme, is adapted to solve a multi-classes hypothesis test problem in our work. With the proposed algorithm, the computational cost is reduced significantly without sacrificing the performance of the underlying system. The experimental results based on the UCF101 data set demonstrated the efficiency of the framework compared to the fixed sampling scheme.
A Systematic Survey on Human Behavior Recognition Methods

Yuan Meixue,Deepsim Intelligence Technology Co.,Zhao Jindong,Sun Ming

DOI: https://doi.org/10.1007/s42979-021-00932-x

2021-01-01

SN Computer Science

Abstract:Human behavior is an essential component of social interaction and is of great significance to identify and analyze human behaviors in a variety of fields. Due to the rapid development of computer vision and machine learning technology, machine with intelligence has started replacing human beings to observe, perceive and analyze the explosive growth of image and video data. Computer vision and machine learning-based human behavior recognition is one of these tasks, which has become a particularly hot research topic in many different fields, such as intelligent monitoring, human–computer interaction, smart home, virtual reality, and medical diagnosis. In this study, we survey systematically the popular methods, algorithms, models and well-known action datasets in human behavior analysis in the past two decades. In addition, the advantages and disadvantages of the methods are discussed and propitious future research directions are also presented. The results of this survey reveal that paradigms of human behavior analysis is being shifted from traditional RGB to RGB-D, from deep learning to more intelligent and automated deep reinforcement learning, and from fixed camera devices to portable devices and channel state information (CSI), and paradigms based on automated deep reinforcement learning and portable devices and CSI would become some hot topics for future research on human behavior analysis.

Recognition of Humans and Their Activities Using Video

Human Action Recognition

Human Action Recognition Using Deep Learning Methods.

Human activity recognition for video surveillance

A Survey Of Activity Recognition And Understanding The Behavior In Video Survelliance

A human activity recognition framework in videos using segmented human subject focus

Video-understanding framework for automatic behavior recognition

Effective approaches in human action recognition

Multi-object Events Recognition from Video Sequences Using Extended Finite State Machine

A recent survey for human activity recoginition based on deep learning approach

Recognition of Human Activities Using Svm Multi-Class Classifier

Real Time Human Action Recognition in a Long Video Sequence

Human behavior understanding for video surveillance: Recent advance

Video-Based Abnormal Human Behavior Recognition - A Review.

Human Interaction Recognition Fusing Multiple Features of Depth Sequences

Human Activity Recognition based on Dynamic Spatio-Temporal Relations

Human behaviour recognition with mid-level representations for crowd understanding and analysis

Human Activity Recognition Based On Video Summarization And Deep Convolutional Neural Network

Fast and Reliable Human Action Recognition in Video Sequences by Sequential Analysis

A Systematic Survey on Human Behavior Recognition Methods