Abstract:Spatial-temporal modeling is crucial for action recognition in videos within the field of artificial intelligence. However, robustly extracting motion information remains a primary challenge due to temporal deformations of appearances and variations in motion frequencies between different actions. In order to address these issues, we propose an innovative and effective method called the Motion Sensitive Network (MSN), incorporating the theories of artificial neural networks and key concepts of autonomous system control and decision-making. Specifically, we employ an approach known as Spatial-Temporal Pyramid Motion Extraction (STP-ME) module, adjusting convolution kernel sizes and time intervals synchronously to gather motion information at different temporal scales, aligning with the learning and prediction characteristics of artificial neural networks. Additionally, we introduce a new module called Variable Scale Motion Excitation (DS-ME), utilizing a differential model to capture motion information in resonance with the flexibility of autonomous system control. Particularly, we employ a multi-scale deformable convolutional network to alter the motion scale of the target object before computing temporal differences across consecutive frames, providing theoretical support for the flexibility of autonomous systems. Temporal modeling is a crucial step in understanding environmental changes and actions within autonomous systems, and MSN, by integrating the advantages of Artificial Neural Networks (ANN) in this task, provides an effective framework for the future utilization of artificial neural networks in autonomous systems. We evaluate our proposed method on three challenging action recognition datasets (Kinetics-400, Something-Something V1, and Something-Something V2). The results indicate an improvement in accuracy ranging from 1.1% to 2.2% on the test set. When compared with state-of-the-art (SOTA) methods, the proposed approach achieves a maximum performance of 89.90%. In ablation experiments, the performance gain of this module also shows an increase ranging from 2% to 5.3%. The introduced Motion Sensitive Network (MSN) demonstrates significant potential in various challenging scenarios, providing an initial exploration into integrating artificial neural networks into the domain of autonomous systems.

MPSN: Motion-aware Pseudo-Siamese Network for indoor video head detection in buildings

MGPSN: Motion-Guided Pseudo Siamese Network for Indoor Video Head Detection.

Towards Accurate Dense Pedestrian Detection Via Occlusion-Prediction Aware Label Assignment and Hierarchical-Nms.

Multi-Source Information Fusion network for building occupancy estimation

Capturing Humans in Motion: Temporal-Attentive 3D Human Pose and Shape Estimation from Monocular Video

DMSC-Net: A deep Multi-Scale context network for 3D object detection of indoor point clouds

Unveiling personnel movement in a larger indoor area with a non-overlapping multi-camera system

Motion sensitive network for action recognition in control and decision-making of autonomous systems

Vision-based indoor occupants detection system for intelligent buildings

PMIndoor: Pose Rectified Network and Multiple Loss Functions for Self-Supervised Monocular Indoor Depth Estimation

Indoor Passive Visual Positioning by CNN-Based Pedestrian Detection

Spatial and Semantic Information Enhancement for Indoor 3D Object Detection

MotionNet: Joint Perception and Motion Prediction for Autonomous Driving Based on Bird's Eye View Maps

Semantic Plane-Structure Based Motion Detection with a Nonstationary Camera

Building Occupancy Detection and Localization Using CCTV Camera and Deep Learning

Wi-Exercise: An Indoor Human Movement Detection Method Based on Bidirectional LSTM Attention

Multi-Object Tracking Algorithm for RGB-D Images Based on Asymmetric Dual Siamese Networks

High-accuracy Occupancy Counting at Crowded Entrances for Smart Buildings

A Video-Based Indoor Occupant Detection And Localization Algorithm For Smart Buildings

Toward Pedestrian Head Tracking: A Benchmark Dataset and an Information Fusion Network

Deep Learning and Hybrid Approaches for Dynamic Scene Analysis, Object Detection and Motion Tracking