Abstract:This paper proposes MambaST, a plug-and-play cross-spectral spatial-temporal fusion pipeline for efficient pedestrian detection. Several challenges exist for pedestrian detection in autonomous driving applications. First, it is difficult to perform accurate detection using RGB cameras under dark or low-light conditions. Cross-spectral systems must be developed to integrate complementary information from multiple sensor modalities, such as thermal and visible cameras, to improve the robustness of the detections. Second, pedestrian detection models are latency-sensitive. Efficient and easy-to-scale detection models with fewer parameters are highly desirable for real-time applications such as autonomous driving. Third, pedestrian video data provides spatial-temporal correlations of pedestrian movement. It is beneficial to incorporate temporal as well as spatial information to enhance pedestrian detection. This work leverages recent advances in the state space model (Mamba) and proposes a novel Multi-head Hierarchical Patching and Aggregation (MHHPA) structure to extract both fine-grained and coarse-grained information from both RGB and thermal imagery. Experimental results show that the proposed MHHPA is an effective and efficient alternative to a Transformer model for cross-spectral pedestrian detection. Our proposed model also achieves superior performance on small-scale pedestrian detection. The code is available at <a class="link-external link-https" href="https://github.com/XiangboGaoBarry/MambaST" rel="external noopener nofollow">this https URL</a>}{<a class="link-external link-https" href="https://github.com/XiangboGaoBarry/MambaST" rel="external noopener nofollow">this https URL</a>.

Flow Mosaicking: Real-Time Pedestrian Counting Without Scene-Specific Learning

A Double-Region Learning Algorithm for Counting the Number of Pedestrians in Subway Surveillance Videos

Counting moving people in crowds using motion statistics of feature-points

Dynamic Region Division for Adaptive Learning Pedestrian Counting

Counting People by Estimating People Flows

Fast Pedestrian Detection And Tracking Based On Vibe Combined Hog-Svm Scheme

Dynamic Video Streaming Real-time Headcount Based on Improved YOLO V5 for Open Plazas

Semisupervised Pedestrian Counting With Temporal and Spatial Consistencies

Robust crowd counting using detection flow

Flux based pedestrian flow estimation in occluded scenes

Deep Fundamental Diagram Network for Real-Time Pedestrian Dynamics Analysis

A Real-Time Pedestrian Counting System Based on RGB-D

Pedestrian detection based on modified dynamic background using gaussian mixture models and hog-svm detection

An Efficient Methodology for Calibrating Traffic Flow Models Based on Bisection Analysis

Predicting Pedestrian Counts in Crowded Scenes With Rich and High-Dimensional Features

Pedestrian Detection by Learning a Mixture Mask Model and Its Implementation

Automatic Identification of Human Subgroups in Time-Dependent Pedestrian Flow Networks

ScaleFlow++: Robust and Accurate Estimation of 3D Motion from Video

MambaST: A Plug-and-Play Cross-Spectral Spatial-Temporal Fuser for Efficient Pedestrian Detection

Fast moving pedestrian detection based on motion segmentation and new motion features

A Flow Base Bi-path Network for Cross-scene Video Crowd Understanding in Aerial View