Abstract:Purpose The purpose of this paper is to provide a fast and accurate network for spatiotemporal action localization in videos. It detects human actions both in time and space simultaneously in real-time, which is applicable in real-world scenarios such as safety monitoring and collaborative assembly. Design/methodology/approach This paper design an end-to-end deep learning network called collaborator only watch once (COWO). COWO recognizes the ongoing human activities in real-time with enhanced accuracy. COWO inherits from the architecture of you only watch once (YOWO), known to be the best performing network for online action localization to date, but with three major structural modifications: COWO enhances the intraclass compactness and enlarges the interclass separability in the feature level. A new correlation channel fusion and attention mechanism are designed based on the Pearson correlation coefficient. Accordingly, a correction loss function is designed. This function minimizes the same class distance and enhances the intraclass compactness. Use a probabilistic K-means clustering technique for selecting the initial seed points. The idea behind this is that the initial distance between cluster centers should be as considerable as possible. CIOU regression loss function is applied instead of the Smooth L1 loss function to help the model converge stably. Findings COWO outperforms the original YOWO with improvements of frame mAP 3% and 2.1% at a speed of 35.12 fps. Compared with the two-stream, T-CNN, C3D, the improvement is about 5% and 14.5% when applied to J-HMDB-21, UCF101-24 and AGOT data sets. Originality/value COWO extends more flexibility for assembly scenarios as it perceives spatiotemporal human actions in real-time. It contributes to many real-world scenarios such as safety monitoring and collaborative assembly.

Efficient Search and Localization of Human Actions in Video Databases

A Method of Simultaneously Action Recognition and Video Segmentation of Video Streams.

Retrieval in Long Surveillance Videos using User Described Motion and Object Attributes

Video retrieval using VQ-based global motion features

Detecting Temporal Proposal for Action Localization with Tree-structured Search Policy

Searching Action Proposals Via Spatial Actionness Estimation And Temporal Path Inference And Tracking

Skimming, Locating, then Perusing: A Human-Like Framework for Natural Language Video Localization

Semantic-based surveillance video retrieval

Localizing Unseen Activities in Video Via Image Query

A Fast Sub-Volume Search Method for Human Action Detection

Semantic Proposal for Activity Localization in Videos Via Sentence Query

Analysis of Human Actions for Video Indexing

Incremetal Spatio-Temporal Feature Extraction and Retrieval for Large Video Database

A Novel Approach to Video Indexing and Retrieval Using Motion Activity

Localizing Volumetric Motion for Action Recognition in Realistic Videos

Probabilistic Semantic Retrieval for Surveillance Videos with Activity Graphs

Local Spatio-Temporal Feature Based Voting Framework for Complex Human Activity Detection and Localization

A Novel System for Video Retrieval of Frontal-View Indoor Moving Pedestrians

COWO: Towards Real-Time Spatiotemporal Action Localization in Videos

An Efficient Approach to Content-Based Object Retrieval in Videos.

Automatic extraction of relevant video shots of specific actions exploiting Web data