Abstract:Object detection and semantic segmentation are two fundamental techniques for Intelligent Vehicles (IV) and Advanced Driving Assistance System (ADAS). Motivated by recent studies demonstrating that object detection and semantic segmentation are two highly-correlated tasks, this paper handles the problem of joint object detection and semantic segmentation in traffic scenes. Existing methods perform the joint object detection and semantic segmentation by sharing the same backbone network, but always ignore the interactive connection between the subdividing detection branch and segmentation branch, leading to the insufficient interaction between the two branches. Considering this situation, this paper proposes a joint object detection and semantic segmentation model with the cross-attention and inner-attention mechanisms. The cross-attention mechanism enables to build up the essential interaction between the subdividing detection branch and segmentation branch to fully make use of their correlation. In addition, the inner-attention contributes to strengthening the representations of feature maps in the model. Given an image, an encoder-decoder network is firstly used to extract initial feature maps. Then, the inner-attention mechanism is applied to strengthen the initial feature maps to obtain segmentation feature maps. Subsequently, the cross-attention mechanism utilizes the segmentation feature maps to guide the generation of object detection feature maps. Finally, the semantic segmentation is performed on the segmentation feature maps and object detection is performed on the detection feature maps. In the experiments, two well-known public traffic datasets are used to evaluate our model. Our model achieves the highest performance in comparison with several recently-proposed methods. In addition, some ablation studies are conducted to evaluate the proposed inner-attention and cross-attention mechanisms, and experiment results validate their effectiveness.

A Cross-Attention and Siamese Network Based Model for Off-Topic Detection

Automatic Depression Prediction Via Cross-Modal Attention-Based Multi-Modal Fusion in Social Networks

A Joint Object Detection and Semantic Segmentation Model with Cross-Attention and Inner-Attention Mechanisms

An Attention Based Model for Off-Topic Spontaneous Spoken Response Detection: an Initial Study.

Off-Topic Detection of Business English Essay Based on Deep Learning Model

Attention-based model for predicting question relatedness on Stack Overflow

TopicAns: Topic-informed Architecture for Answer Recommendation on Technical Q&A Site

A Local Self-Attention Sentence Model for Answer Selection Task in CQA Systems

ProtSi: Prototypical Siamese Network with Data Augmentation for Few-Shot Subjective Answer Evaluation

An automatic short-answer grading model for semi-open-ended questions

Detecting Offensive Language on Social Networks: an End-to-end Detection Method Based on Graph Attention Networks

A Network Intrusion Detection Model Based on BiLSTM with Multi-Head Attention Mechanism

A Multi-Size Neural Network with Attention Mechanism for Answer Selection

aNMM: Ranking Short Answer Texts with Attention-Based Neural Matching Model

Chinese Sentences Similarity via Cross-Attention Based Siamese Network

Optimizing Automatic Text Classification Approach in Adaptive Online Collaborative Discussion–A Perspective of Attention Mechanism-Based Bi-LSTM

A Context-focused Attention Evolution Model for Aspect-based Sentiment Classification

Multimodality Sentiment Analysis in Social Internet of Things Based on Hierarchical Attentions and CSAT-TCN with MBM Network.

Joint modeling method of question intent detection and slot filling for domain-oriented question answering system

An Unsupervised Approach for Low-Quality Answer Detection in Community Question-Answering

An Enhanced Convolutional Neural Network Model for Answer Selection.