Abstract:In complex traffic scenes, the orientation and location of pedestrians are important criteria for judging their intentions. We note that pedestrians are characterized by variability in appearance and small differences among orientations (especially adjacent orientations), thus causing general object detection algorithms to perform poorly in extracting features. So, extracting more discriminative features is an effective way to solve this problem. To this end, we propose a novel framework to enhance feature extraction involving pedestrian orientation detection (orientation classification and location regression). The framework consists of two modules, multi-window Transformer parallel fusion feature pyramid (MTPF) and gated graph (GG). The MTPF module is used for multi-layer feature fusion, which improves the feature representation of the prediction map by extracting high-level semantic information from deep layers and recovering missing contextual information from shallow layers. Specifically, it is achieved by setting a sliding window on multiple prediction maps and fused by the Transformer. The region proposal is abstracted into a graph with six nodes in the GG module, where each node represents a body part. We utilize GG to learn the spatial dependencies among body parts and learn features by aggregating information from neighbors. Finally, pedestrian orientation classification and location regression are performed on a graph containing rich relationships among nodes. According to the survey, there are currently no methods and datasets that can be directly used for pedestrian orientation detection, so we manually annotate pedestrian orientations on three public datasets containing a large number of pedestrian samples, and compare the proposed method with the current state-of-the-art object detection methods by comparison, the results demonstrate the effectiveness of the proposed method.

Transformer fusion and histogram layer multispectral pedestrian detection network

Locality guided cross-modal feature aggregation and pixel-level fusion for multispectral pedestrian detection

Cross-modality feature fusion for night pedestrian detection

Deep saliency detection-based pedestrian detection with multispectral multi-scale features fusion network

TS-Net: two-stream network based on multispectral feature fusion

Optimal Fusion-based Asymmetric Two-stream Networks for Multispectral Image Pedestrian Detection

Robust Pedestrian Detection Based on Multi-Spectral Image Fusion and Convolutional Neural Networks

TF-YOLO: A Transformer–Fusion-Based YOLO Detector for Multimodal Pedestrian Detection in Autonomous Driving Scenes

Multispectral Deep Neural Networks for Pedestrian Detection

Cascaded information enhancement and cross-modal attention feature fusion for multispectral pedestrian detection

MS-DETR: Multispectral Pedestrian Detection Transformer with Loosely Coupled Fusion and Modality-Balanced Optimization

A Fast RetinaNet Fusion Framework for Multi-Spectral Pedestrian Detection

TFDet: Target-Aware Fusion for RGB-T Pedestrian Detection

Pedestrian detection-driven cascade network for infrared and visible image fusion

Improving Multispectral Pedestrian Detection with Scale‐aware Permutation Attention and Adjacent Feature Aggregation

Exploiting fusion architectures for multispectral pedestrian detection and segmentation

Multiscale Cross-modal Homogeneity Enhancement and Confidence-aware Fusion for Multispectral Pedestrian Detection

Cross-modality interactive attention network for multispectral pedestrian detection.

Multi-window Transformer Parallel Fusion Feature Pyramid Network for Pedestrian Orientation Detection

Fusion of Multispectral Data Through Illumination-aware Deep Neural Networks for Pedestrian Detection