Abstract:Object detection and semantic segmentation are two fundamental techniques for Intelligent Vehicles (IV) and Advanced Driving Assistance System (ADAS). Motivated by recent studies demonstrating that object detection and semantic segmentation are two highly-correlated tasks, this paper handles the problem of joint object detection and semantic segmentation in traffic scenes. Existing methods perform the joint object detection and semantic segmentation by sharing the same backbone network, but always ignore the interactive connection between the subdividing detection branch and segmentation branch, leading to the insufficient interaction between the two branches. Considering this situation, this paper proposes a joint object detection and semantic segmentation model with the cross-attention and inner-attention mechanisms. The cross-attention mechanism enables to build up the essential interaction between the subdividing detection branch and segmentation branch to fully make use of their correlation. In addition, the inner-attention contributes to strengthening the representations of feature maps in the model. Given an image, an encoder-decoder network is firstly used to extract initial feature maps. Then, the inner-attention mechanism is applied to strengthen the initial feature maps to obtain segmentation feature maps. Subsequently, the cross-attention mechanism utilizes the segmentation feature maps to guide the generation of object detection feature maps. Finally, the semantic segmentation is performed on the segmentation feature maps and object detection is performed on the detection feature maps. In the experiments, two well-known public traffic datasets are used to evaluate our model. Our model achieves the highest performance in comparison with several recently-proposed methods. In addition, some ablation studies are conducted to evaluate the proposed inner-attention and cross-attention mechanisms, and experiment results validate their effectiveness.

Anchor-free pedestrain detection model with semantic context of traffic scenario

See Extensively While Focusing on the Core Area for Pedestrian Detection.

Towards Accurate Dense Pedestrian Detection Via Occlusion-Prediction Aware Label Assignment and Hierarchical-Nms.

Improved SENet for Pedestrian Detection

Pedestrian As Points: an Improved Anchor-Free Method for Center-Based Pedestrian Detection.

A Joint Object Detection and Semantic Segmentation Model with Cross-Attention and Inner-Attention Mechanisms

Putting the Anchors Efficiently: Geometric Constrained Pedestrian Detection.

Pedestrian Detection via Body Part Semantic and Contextual Information With DNN

An anchor-free detector and R-CNN integrated neural network architecture for environmental perception of urban roads

Anchor-free infrared pedestrian detection based on cross-scale feature fusion and hierarchical attention mechanism

CFRLA-Net: A Context-aware Feature Representation Learning Anchor-free Network for Pedestrian Detection

SPV-SSD: An Anchor-Free 3D Single-Stage Detector with Supervised-PointRendering and Visibility Representation

Anchor Free Multispectral Pedestrian Detection Algorithm Based on Differential Feature Attention Mechanism

Pedestrian Detection aided by Deep Learning Semantic Tasks

SurfaceNet: A Surface Focused Network for Pedestrian Detection and Segmentation in 3D Point Clouds

Pedestrian Intention Prediction Based on Traffic-Aware Scene Graph Model.

A Multi-Branch Anchor-Free Detection Algorithm for Hospital Pedestrian

AspectNet: Aspect-Aware Anchor-Free Detector for Autonomous Driving

Improving Small-Scale Pedestrian Detection Using Informed Context

Research on Deep Learning Detection Model for Pedestrian Objects in Complex Scenes Based on Improved YOLOv7

Occluded Pedestrian Attention Network : an Occluded Pedestrian Detector