Abstract:Object detection and semantic segmentation are two fundamental techniques for Intelligent Vehicles (IV) and Advanced Driving Assistance System (ADAS). Motivated by recent studies demonstrating that object detection and semantic segmentation are two highly-correlated tasks, this paper handles the problem of joint object detection and semantic segmentation in traffic scenes. Existing methods perform the joint object detection and semantic segmentation by sharing the same backbone network, but always ignore the interactive connection between the subdividing detection branch and segmentation branch, leading to the insufficient interaction between the two branches. Considering this situation, this paper proposes a joint object detection and semantic segmentation model with the cross-attention and inner-attention mechanisms. The cross-attention mechanism enables to build up the essential interaction between the subdividing detection branch and segmentation branch to fully make use of their correlation. In addition, the inner-attention contributes to strengthening the representations of feature maps in the model. Given an image, an encoder-decoder network is firstly used to extract initial feature maps. Then, the inner-attention mechanism is applied to strengthen the initial feature maps to obtain segmentation feature maps. Subsequently, the cross-attention mechanism utilizes the segmentation feature maps to guide the generation of object detection feature maps. Finally, the semantic segmentation is performed on the segmentation feature maps and object detection is performed on the detection feature maps. In the experiments, two well-known public traffic datasets are used to evaluate our model. Our model achieves the highest performance in comparison with several recently-proposed methods. In addition, some ablation studies are conducted to evaluate the proposed inner-attention and cross-attention mechanisms, and experiment results validate their effectiveness.

Channel and spatial attention based deep object co-segmentation

Semantic Aware Attention Based Deep Object Co-segmentation

Deep Dual-Stream Network with Scale Context Selection Attention Module for Semantic Segmentation

Deep Object Co-segmentation via Spatial-Semantic Network Modulation

Group-wise Deep Object Co-Segmentation with Co-Attention Recurrent Neural Network

See More, Know More: Unsupervised Video Object Segmentation with Co-Attention Siamese Networks

Human Centred Object Co-Segmentation

Channelized Axial Attention for Semantic Segmentation -- Considering Channel Relation within Spatial Attention for Semantic Segmentation

Semantic Image Segmentation with Improved Position Attention and Feature Fusion

A Joint Object Detection and Semantic Segmentation Model with Cross-Attention and Inner-Attention Mechanisms

CAA : Channelized Axial Attention for Semantic Segmentation.

Learning Cross-Channel Representations for Semantic Segmentation

SCSA: Exploring the Synergistic Effects Between Spatial and Channel Attention

Zero-Shot Video Object Segmentation with Co-Attention Siamese Networks

Research on Image Semantic Segmentation Based on Hybrid Cascade Feature Fusion and Detailed Attention Mechanism

Semantic Image Segmentation Based On Attentions To Intra Scales And Inner Channels

A Synergistical Attention Model for Semantic Segmentation of Remote Sensing Images

Attention-Guided Multi-Scale Fusion Network for Similar Objects Semantic Segmentation

Adversarial domain adaptation with Siamese network for video object cosegmentation

Toward Stable Co-Saliency Detection and Object Co-Segmentation

LCCo: Lending CLIP to Co-Segmentation