Abstract:Spatial-temporal modeling is crucial for action recognition in videos within the field of artificial intelligence. However, robustly extracting motion information remains a primary challenge due to temporal deformations of appearances and variations in motion frequencies between different actions. In order to address these issues, we propose an innovative and effective method called the Motion Sensitive Network (MSN), incorporating the theories of artificial neural networks and key concepts of autonomous system control and decision-making. Specifically, we employ an approach known as Spatial-Temporal Pyramid Motion Extraction (STP-ME) module, adjusting convolution kernel sizes and time intervals synchronously to gather motion information at different temporal scales, aligning with the learning and prediction characteristics of artificial neural networks. Additionally, we introduce a new module called Variable Scale Motion Excitation (DS-ME), utilizing a differential model to capture motion information in resonance with the flexibility of autonomous system control. Particularly, we employ a multi-scale deformable convolutional network to alter the motion scale of the target object before computing temporal differences across consecutive frames, providing theoretical support for the flexibility of autonomous systems. Temporal modeling is a crucial step in understanding environmental changes and actions within autonomous systems, and MSN, by integrating the advantages of Artificial Neural Networks (ANN) in this task, provides an effective framework for the future utilization of artificial neural networks in autonomous systems. We evaluate our proposed method on three challenging action recognition datasets (Kinetics-400, Something-Something V1, and Something-Something V2). The results indicate an improvement in accuracy ranging from 1.1% to 2.2% on the test set. When compared with state-of-the-art (SOTA) methods, the proposed approach achieves a maximum performance of 89.90%. In ablation experiments, the performance gain of this module also shows an increase ranging from 2% to 5.3%. The introduced Motion Sensitive Network (MSN) demonstrates significant potential in various challenging scenarios, providing an initial exploration into integrating artificial neural networks into the domain of autonomous systems.

Supervised Multiple Timescale Recurrent Neuron Network Model for Human Action Classification.

Real-time human action classification using a dynamic neural model.

Continuous Motion Recognition Using Multiple Time Constant Recurrent Neural Network With A Deep Network Model

Multiple Timescale Recurrent Neural Network with Slow Feature Analysis for Efficient Motion Recognition.

Continuous Timescale Long-Short Term Memory Neural Network for Human Intent Understanding

A Fast Training Algorithm of Multiple-Timescale Recurrent Neural Network for Agent Motion Generation

Human motion based intent recognition using a deep dynamic neural model

Recognition of Visually Perceived Compositional Human Actions by Multiple Spatio-Temporal Scales Recurrent Neural Networks

Human Intention Understanding Based On Object Affordance And Action Classification

Online Action Detection and Forecast Via Multitask Deep Recurrent Neural Networks

A Spatio-Temporal Transformer Network for Human Motion Prediction in Human-Robot Collaboration

Multi-Level Recurrent Residual Networks for Action Recognition

Multi‐mode Neural Network for Human Action Recognition

Temporal channel reconfiguration multi‐graph convolution network for skeleton‐based action recognition

Predictive Coding for Dynamic Vision : Development of Functional Hierarchy in a Multiple Spatio-Temporal Scales RNN Model

Multi-Scale Proposal Regression Network for Temporal Action Proposal Generation

Multi-Branch Spatial-Temporal Network for Action Recognition

Human Activity Recognition Based on Deep-Temporal Learning Using Convolution Neural Networks Features and Bidirectional Gated Recurrent Unit With Features Selection

Time-Aware Multi-Scale RNNs for Time Series Modeling

Motion sensitive network for action recognition in control and decision-making of autonomous systems

DMS-GCN: Dynamic Mutiscale Spatiotemporal Graph Convolutional Networks for Human Motion Prediction