Abstract:Object detection and semantic segmentation are two fundamental techniques for Intelligent Vehicles (IV) and Advanced Driving Assistance System (ADAS). Motivated by recent studies demonstrating that object detection and semantic segmentation are two highly-correlated tasks, this paper handles the problem of joint object detection and semantic segmentation in traffic scenes. Existing methods perform the joint object detection and semantic segmentation by sharing the same backbone network, but always ignore the interactive connection between the subdividing detection branch and segmentation branch, leading to the insufficient interaction between the two branches. Considering this situation, this paper proposes a joint object detection and semantic segmentation model with the cross-attention and inner-attention mechanisms. The cross-attention mechanism enables to build up the essential interaction between the subdividing detection branch and segmentation branch to fully make use of their correlation. In addition, the inner-attention contributes to strengthening the representations of feature maps in the model. Given an image, an encoder-decoder network is firstly used to extract initial feature maps. Then, the inner-attention mechanism is applied to strengthen the initial feature maps to obtain segmentation feature maps. Subsequently, the cross-attention mechanism utilizes the segmentation feature maps to guide the generation of object detection feature maps. Finally, the semantic segmentation is performed on the segmentation feature maps and object detection is performed on the detection feature maps. In the experiments, two well-known public traffic datasets are used to evaluate our model. Our model achieves the highest performance in comparison with several recently-proposed methods. In addition, some ablation studies are conducted to evaluate the proposed inner-attention and cross-attention mechanisms, and experiment results validate their effectiveness.

Biologically Motivated Object Detection Based On Integration Of Top-Down And Bottom-Up Attention

Top-down attention guided object detection

A Visual Attention Based Object Detection Model beyond Top-Down and Bottom-up Mechanism

A visual attention model combining top-down and bottom-up mechanisms for salient object detection

A Biologically Inspired Object-Based Visual Attention Model

EBiDA-FPN: Enhanced Bi-Directional Attention Feature Pyramid Network for Object Detection

Efficient Object Detection Based on Selective Attention

Object Detection for Remote Sensing Image Based on Top-Down Visual Attention

Object Detection Method Based on"What"and"Where"Information

Integration of Bottom-Up and Top-Down Cues in Bayesian Network for Object Detection

Object detection based on an adaptive attention mechanism

Action-Driven Object Detection with Top-Down Visual Attentions

A Computational Model of Object-Based Attentionusing Multi-scale Analysis and Grouping

Local Attention Sequence Model for Video Object Detection

Improvement of Salient-Region Detection Using an Integrated Bottom-Up Model

A Salient Object Detection Framework Beyond Top-Down and Bottom-Up Mechanism

Visual Attention Model with a Novel Learning Strategy and Its Application to Target Detection from SAR Images.

A Joint Object Detection and Semantic Segmentation Model with Cross-Attention and Inner-Attention Mechanisms

MLA-Net: Feature Pyramid Network with Multi-Level Local Attention for Object Detection

A Biologically-Inspired Top-Down Learning Model Based on Visual Attention

“What” and “Where” Information Based Attention Guidance Model