Abstract:Spatial-temporal modeling is crucial for action recognition in videos within the field of artificial intelligence. However, robustly extracting motion information remains a primary challenge due to temporal deformations of appearances and variations in motion frequencies between different actions. In order to address these issues, we propose an innovative and effective method called the Motion Sensitive Network (MSN), incorporating the theories of artificial neural networks and key concepts of autonomous system control and decision-making. Specifically, we employ an approach known as Spatial-Temporal Pyramid Motion Extraction (STP-ME) module, adjusting convolution kernel sizes and time intervals synchronously to gather motion information at different temporal scales, aligning with the learning and prediction characteristics of artificial neural networks. Additionally, we introduce a new module called Variable Scale Motion Excitation (DS-ME), utilizing a differential model to capture motion information in resonance with the flexibility of autonomous system control. Particularly, we employ a multi-scale deformable convolutional network to alter the motion scale of the target object before computing temporal differences across consecutive frames, providing theoretical support for the flexibility of autonomous systems. Temporal modeling is a crucial step in understanding environmental changes and actions within autonomous systems, and MSN, by integrating the advantages of Artificial Neural Networks (ANN) in this task, provides an effective framework for the future utilization of artificial neural networks in autonomous systems. We evaluate our proposed method on three challenging action recognition datasets (Kinetics-400, Something-Something V1, and Something-Something V2). The results indicate an improvement in accuracy ranging from 1.1% to 2.2% on the test set. When compared with state-of-the-art (SOTA) methods, the proposed approach achieves a maximum performance of 89.90%. In ablation experiments, the performance gain of this module also shows an increase ranging from 2% to 5.3%. The introduced Motion Sensitive Network (MSN) demonstrates significant potential in various challenging scenarios, providing an initial exploration into integrating artificial neural networks into the domain of autonomous systems.

Energy efficiency and sensitivity benefits in a motion processing adaptive recurrent neural network

Adaptive V1-MT Model for Motion Perception

Modelling Human Visual Motion Processing with Trainable Motion Energy Sensing and a Self-attention Network

How Deep Neural Networks Understand Motion? Toward Interpretable Motion Modeling by Leveraging the Relative Change in Position

Biophysical neural adaptation mechanisms enable artificial neural networks to capture dynamic retinal computation

Adaptation optimizes sensory encoding of future stimuli

A spatiotemporal energy model based on spiking neurons for human motion perception

Pre-training artificial neural networks with spontaneous retinal activity improves motion prediction in natural scenes

Recurrent network dynamics reconciles visual motion segmentation and integration

Joint contribution of adaptation and neuronal population recruitment to response level in visual area MT: a computational model

Adaptive coding efficiency in recurrent cortical circuits via gain control

Spatio-Temporal Adaptation in the Unsupervised Development of Networked Visual Neurons

Motion sensitive network for action recognition in control and decision-making of autonomous systems

Artificial Visual Network with Fully Modeled Retinal Direction-Selective Neural Pathway for Motion Direction Detection in Grayscale Scenes

Adaptation in the Visual System: Networked Fatigue or Suppressed Prediction Error Signalling?

Deep predictive coding networks partly capture neural signatures of short-term temporal adaptation in human visual cortex

Adaptation shapes the representational geometry in mouse V1 to efficiently encode the environment

Hierarchical temporal prediction captures motion processing from retina to higher visual cortex

Adaptation of retinal discriminability to natural scenes

Contrast-dependent response modulation in convolutional neural networks captures behavioral and neural signatures of visual adaptation

Cascaded effects of spatial adaptation in the early visual system