Abstract:This paper proposes MambaST, a plug-and-play cross-spectral spatial-temporal fusion pipeline for efficient pedestrian detection. Several challenges exist for pedestrian detection in autonomous driving applications. First, it is difficult to perform accurate detection using RGB cameras under dark or low-light conditions. Cross-spectral systems must be developed to integrate complementary information from multiple sensor modalities, such as thermal and visible cameras, to improve the robustness of the detections. Second, pedestrian detection models are latency-sensitive. Efficient and easy-to-scale detection models with fewer parameters are highly desirable for real-time applications such as autonomous driving. Third, pedestrian video data provides spatial-temporal correlations of pedestrian movement. It is beneficial to incorporate temporal as well as spatial information to enhance pedestrian detection. This work leverages recent advances in the state space model (Mamba) and proposes a novel Multi-head Hierarchical Patching and Aggregation (MHHPA) structure to extract both fine-grained and coarse-grained information from both RGB and thermal imagery. Experimental results show that the proposed MHHPA is an effective and efficient alternative to a Transformer model for cross-spectral pedestrian detection. Our proposed model also achieves superior performance on small-scale pedestrian detection. The code is available at <a class="link-external link-https" href="https://github.com/XiangboGaoBarry/MambaST" rel="external noopener nofollow">this https URL</a>}{<a class="link-external link-https" href="https://github.com/XiangboGaoBarry/MambaST" rel="external noopener nofollow">this https URL</a>.

Pedestrian Detection Using Sparse Gabor Filter and Support Vector Machine

A Novel Approach to Design the Fast Pedestrian Detection for Video Surveillance System

Research on Hierarchical Pedestrian Detection Based on SVM Classifier with Improved Kernel Function

Pedestrian detection based on modified dynamic background using gaussian mixture models and hog-svm detection

Fast Pedestrian Detection And Tracking Based On Vibe Combined Hog-Svm Scheme

Pedestrian Detection in Complex Scene Using Full Binary Tree Classifiers Based on Locally Assembled Binary Haar-Like Features

Representation and Analysis of Pedestrian Crossing States Based on Video Tracking

A Monocular Vision Based Pedestrian Detection System for Intelligent Vehicles

Feature Selection And Pedestrian Detection Based On Sparse Representation

Pedestrian Detection Using Hybrid Statistical Feature.

Pedestrian movement trajectory reappearance and crossing feature expression based on video processing

Pedestrian detection with unsupervised multispectral feature learning using deep neural networks

Effective Pedestrian Detection Using Center-symmetric Local Binary/Trinary Patterns

Robust Pedestrian Classification Based on Hierarchical Kernel Sparse Representation

Pedestrian Detection Based on Sparse Coding and Transfer Learning

Pedestrian Detection Using a Mixture Mask Model

A Filter Module Used In Pedestrian Detection System

Pedestrian Detection by Learning a Mixture Mask Model and Its Implementation

Real-Time Pedestrian-Vehicle Conflict Detection Algorithm Using Video Data

The study of the detection of pedestrian and bicycle using image processing

MambaST: A Plug-and-Play Cross-Spectral Spatial-Temporal Fuser for Efficient Pedestrian Detection