Abstract:Object detection and semantic segmentation are two fundamental techniques for Intelligent Vehicles (IV) and Advanced Driving Assistance System (ADAS). Motivated by recent studies demonstrating that object detection and semantic segmentation are two highly-correlated tasks, this paper handles the problem of joint object detection and semantic segmentation in traffic scenes. Existing methods perform the joint object detection and semantic segmentation by sharing the same backbone network, but always ignore the interactive connection between the subdividing detection branch and segmentation branch, leading to the insufficient interaction between the two branches. Considering this situation, this paper proposes a joint object detection and semantic segmentation model with the cross-attention and inner-attention mechanisms. The cross-attention mechanism enables to build up the essential interaction between the subdividing detection branch and segmentation branch to fully make use of their correlation. In addition, the inner-attention contributes to strengthening the representations of feature maps in the model. Given an image, an encoder-decoder network is firstly used to extract initial feature maps. Then, the inner-attention mechanism is applied to strengthen the initial feature maps to obtain segmentation feature maps. Subsequently, the cross-attention mechanism utilizes the segmentation feature maps to guide the generation of object detection feature maps. Finally, the semantic segmentation is performed on the segmentation feature maps and object detection is performed on the detection feature maps. In the experiments, two well-known public traffic datasets are used to evaluate our model. Our model achieves the highest performance in comparison with several recently-proposed methods. In addition, some ablation studies are conducted to evaluate the proposed inner-attention and cross-attention mechanisms, and experiment results validate their effectiveness.

Weakly Supervised Fixated Object Detection in Traffic Videos Based on Driver’s Selective Attention Mechanism

A Fusion Method Aiming at Environmental Perception of Autonomous Vehicle Based on Visual Scheme

Object-Level Attention Prediction for Drivers in the Information-Rich Traffic Environment

Visual attention computation in video of driving environment

A Joint Object Detection and Semantic Segmentation Model with Cross-Attention and Inner-Attention Mechanisms

DoTA: Unsupervised Detection of Traffic Anomaly in Driving Videos

Where and What: Driver Attention-based Object Detection

Text-Driven Traffic Anomaly Detection with Temporal High-Frequency Modeling in Driving Videos

Improving real-time driver distraction detection via constrained attention mechanism

Small Object Detection in Traffic Scenes Based on Attention Feature Fusion

A visual attention based method for detecting traffic signs of interest

VSSA-NET: Vertical Spatial Sequence Attention Network for Traffic Sign Detection

DADA-2000: Can Driving Accident be Predicted by Driver Attention? Analyzed by A Benchmark

TAD: A Large-Scale Benchmark for Traffic Accidents Detection from Video Surveillance

On-Road Object Importance Estimation: A New Dataset and A Model with Multi-Fold Top-Down Guidance

All in One Network for Driver Attention Monitoring

A Memory-Augmented Multi-Task Collaborative Framework for Unsupervised Traffic Accident Detection in Driving Videos

Multi-scale feature fusion with attention mechanism for crowded road object detection

Real-Time Vehicle Object Detection Method Based on Multi-Scale Feature Fusion

Toward Effective Traffic Sign Detection via Two-Stage Fusion Neural Networks

Research on Fatigued-Driving Detection Method by Integrating Lightweight YOLOv5s and Facial 3D Keypoints