Abstract:This paper proposes MambaST, a plug-and-play cross-spectral spatial-temporal fusion pipeline for efficient pedestrian detection. Several challenges exist for pedestrian detection in autonomous driving applications. First, it is difficult to perform accurate detection using RGB cameras under dark or low-light conditions. Cross-spectral systems must be developed to integrate complementary information from multiple sensor modalities, such as thermal and visible cameras, to improve the robustness of the detections. Second, pedestrian detection models are latency-sensitive. Efficient and easy-to-scale detection models with fewer parameters are highly desirable for real-time applications such as autonomous driving. Third, pedestrian video data provides spatial-temporal correlations of pedestrian movement. It is beneficial to incorporate temporal as well as spatial information to enhance pedestrian detection. This work leverages recent advances in the state space model (Mamba) and proposes a novel Multi-head Hierarchical Patching and Aggregation (MHHPA) structure to extract both fine-grained and coarse-grained information from both RGB and thermal imagery. Experimental results show that the proposed MHHPA is an effective and efficient alternative to a Transformer model for cross-spectral pedestrian detection. Our proposed model also achieves superior performance on small-scale pedestrian detection. The code is available at <a class="link-external link-https" href="https://github.com/XiangboGaoBarry/MambaST" rel="external noopener nofollow">this https URL</a>}{<a class="link-external link-https" href="https://github.com/XiangboGaoBarry/MambaST" rel="external noopener nofollow">this https URL</a>.

Combining background subtraction and temporal persistency in pedestrian detection from static videos

Pedestrian detection based on modified dynamic background using gaussian mixture models and hog-svm detection

Pedestrian Detection Method Based on Improved YOLOv5s for Densely Occluded Scenarios

A Novel Approach to Design the Fast Pedestrian Detection for Video Surveillance System

Fast Pedestrian Detection And Tracking Based On Vibe Combined Hog-Svm Scheme

Pedestrian Detection in Complex Scene Using Full Binary Tree Classifiers Based on Locally Assembled Binary Haar-Like Features

Locality guided cross-modal feature aggregation and pixel-level fusion for multispectral pedestrian detection

Pedestrian detection using improved Histogram of Oriented Gradients

Taking a Look at Small-Scale Pedestrians and Occluded Pedestrians

Pedestrian Detection Based on an Assembly of Sub-image Detectors

Near-Online Multi-Pedestrian Tracking via Combining Multiple Consistent Appearance Cues

Vision Based Multi-pedestrian Tracking Using Adaptive Detection and Clustering.

Pedestrian detection in video of outdoor condition

Context feature fusion and enhanced non-maximum suppression for pedestrian detection in crowded scenes

An Efficient Pedestrian Detection for Realtime Surveillance Systems Based on Modified YOLOv3

The Improved Pedestrian Target Detection and Re-recognition Algorithm

Cross-Video Pedestrian Tracking Algorithm with a Coordinate Constraint

Effective multiple pedestrian tracking system in video surveillance with monocular stationary camera

Pedestrian Detection Based on Sparse Coding and Transfer Learning

MambaST: A Plug-and-Play Cross-Spectral Spatial-Temporal Fuser for Efficient Pedestrian Detection

An Enhanced Histogram of Oriented Gradients for Pedestrian Detection