Abstract:Object detection and semantic segmentation are two fundamental techniques for Intelligent Vehicles (IV) and Advanced Driving Assistance System (ADAS). Motivated by recent studies demonstrating that object detection and semantic segmentation are two highly-correlated tasks, this paper handles the problem of joint object detection and semantic segmentation in traffic scenes. Existing methods perform the joint object detection and semantic segmentation by sharing the same backbone network, but always ignore the interactive connection between the subdividing detection branch and segmentation branch, leading to the insufficient interaction between the two branches. Considering this situation, this paper proposes a joint object detection and semantic segmentation model with the cross-attention and inner-attention mechanisms. The cross-attention mechanism enables to build up the essential interaction between the subdividing detection branch and segmentation branch to fully make use of their correlation. In addition, the inner-attention contributes to strengthening the representations of feature maps in the model. Given an image, an encoder-decoder network is firstly used to extract initial feature maps. Then, the inner-attention mechanism is applied to strengthen the initial feature maps to obtain segmentation feature maps. Subsequently, the cross-attention mechanism utilizes the segmentation feature maps to guide the generation of object detection feature maps. Finally, the semantic segmentation is performed on the segmentation feature maps and object detection is performed on the detection feature maps. In the experiments, two well-known public traffic datasets are used to evaluate our model. Our model achieves the highest performance in comparison with several recently-proposed methods. In addition, some ablation studies are conducted to evaluate the proposed inner-attention and cross-attention mechanisms, and experiment results validate their effectiveness.

Curiosity-Driven Attention for Anomaly Road Obstacles Segmentation in Autonomous Driving

Segmenting Remote Sensing Anomalies at Instance-level Via Anomaly Map Guided Adaptation

A Novel Method for Road Anomaly Objects Detection in the Traffic Environment With Multi-Mechanism Fusion

Evolving into a Transformer: from a Training-Free Retrieval-Based Method for Anomaly Obstacle Segmentation.

Anomaly-Aware Semantic Segmentation by Leveraging Synthetic-Unknown Data

A Joint Object Detection and Semantic Segmentation Model with Cross-Attention and Inner-Attention Mechanisms

SegmentMeIfYouCan: A Benchmark for Anomaly Segmentation

Latency-aware Road Anomaly Segmentation in Videos: A Photorealistic Dataset and New Metrics

Semantic segmentation of autonomous driving scenes based on multi-scale adaptive attention mechanism

Anomaly-Aware Semantic Segmentation via Style-Aligned OoD Augmentation

Unsupervised Road Anomaly Detection with Language Anchors.

Self-Supervised Likelihood Estimation with Energy Guidance for Anomaly Segmentation in Urban Scenes

Object-Guided and Motion-Refined Attention Network for Video Anomaly Detection

Progressive Temporal-Spatial-Semantic Analysis of Driving Anomaly Detection and Recounting

Efficient Outdoor 3D Point Cloud Semantic Segmentation for Critical Road Objects and Distributed Contexts

UGainS: Uncertainty Guided Anomaly Instance Segmentation

Anomaly Discovery in Semantic Segmentation via Distillation Comparison Networks

Improving real-time driver distraction detection via constrained attention mechanism

Scene Semantic Anomaly Detection of Multi-Sensor in Autonomous Driving

Uncertainty-Aware Point-Cloud Semantic Segmentation for Unstructured Roads

Where Can We Help? A Visual Analytics Approach to Diagnosing and Improving Semantic Segmentation of Movable Objects