Abstract:The booming self-driving cars need to understand the behaviors of other road users for better performance. Recognizing pedestrians' crossing intentions is one of the most critical capabilities of self-driving cars to guarantee safe operations in the urban environment. Some researchers try to predict the future trajectories of pedestrians to avoid a potential collision. Others extract skeleton features from pedestrians to detect specific actions related to the crossing intention. All these methods either neglect the abundant appearance information of pedestrians, or work poorly under severe conditions, e.g., pedestrians standing far away, dim light, and occlusions. In this work, a pedestrian crossing intention recognition (PCIR) framework is proposed to recognize pedestrians' crossing intention. A module for searching targets of interest is introduced to find the pedestrians who are likely to cross the streets, and perform scene perception simultaneously. An action recognition module utilizes a 3D convolutional neural network (CNN) to automatically extract spatiotemporal features that imply early actions before crossing, like limb movements. A distance encoding module makes full use of the contextual cues, e.g., distances between pedestrians and the oncoming vehicle, local traffic scenes around pedestrians, and vehicle speeds, to improve the recognition accuracy obtained from the action recognition module only. Finally, the PCIR framework fuses these factors to predict pedestrians' crossing intentions by minimizing a focal loss. Experimental evaluations verify the effectiveness of the proposed method in recognizing pedestrians' crossing intentions. Comparisons with the skeleton-based methods reveal the robustness of the PCIR framework in the urban environment.

Vision-based Recognition of Pedestrian Crossing Intention in an Urban Environment

Crossing or Not? Context-Based Recognition of Pedestrian Crossing Intention in the Urban Environment

Real-time Pedestrian Crossing Lights Detection Algorithm for the Visually Impaired

Intention Recognition of Pedestrians and Cyclists by 2D Pose Estimation

Coupling Intent and Action for Pedestrian Crossing Behavior Prediction

Pedestrian Crossing Intention Prediction From Surveillance Videos for Over-the-Horizon Safety Warning

Pedestrian Crossing Intention Forecasting at Unsignalized Intersections Using Naturalistic Trajectories

Vision-Based Pedestrian Abnormal Crossing Behavior Recognition and Analysis

Crossing-Road Pedestrian Trajectory Prediction Based on Intention and Behavior Identification

Pedestrian Crossing Intention Prediction Based on Cross-Modal Transformer and Uncertainty-Aware Multi-Task Learning for Autonomous Driving

Pedestrian Detection and Behavior Recognition Based on Vision.

Understanding Pedestrians’ Car-Hailing Intention in Traffic Scenes

Pedestrians Crossing Intention Anticipation Based on Dual‐channel Action Recognition and Hierarchical Environmental Context

Research Of Pedestrian Detection For Intelligent Vehicle Based On Machine Vision

Pedestrian Facial Attention Detection Using Deep Fusion and Multi-modal Fusion Classifier

Intersection Perception Through Real-Time Semantic Segmentation to Assist Navigation of Visually Impaired Pedestrians

Detection of Pedestrians for Far-Infrared Automotive Night Vision Systems Using Learning-Based Method and Head Validation

Pedestrian Crossing Action Recognition and Trajectory Prediction with 3D Human Keypoints

Recognition of Pedestrians’ Crossing Intentions with a Conv-Transformer Network

Pedestrian Crossing Prediction Based on Invariant Feature Extraction of Cross-Spectral Images

Predicting Pedestrian Crossing Intention with Feature Fusion and Spatio-Temporal Attention