Abstract:Multi-modal 3D object detection is instrumental in identifying and localizing objects within 3D space. It combines RGB images from cameras and point-clouds data from lidar sensors, serving as a fundamental technology for autonomous driving applications. Current methods commonly employ simplistic element-wise additions or multiplications to aggregate multi-modal features extracted from point-clouds and images. While these methods enhance detection accuracy, the utilization of basic operations presents challenges in effectively balancing the significance between modalities. This can potentially introduce noise and irrelevant information during the feature aggregation process. Additionally, the multi-level features extracted from images display imbalances in receptive fields. To tackle the aforementioned challenges, we propose two innovative networks: a cross-modality balance network (CMN) and a cross-scale balance network (CSN). CMN incorporates cross-modality attention mechanisms and introduces an auxiliary 2D detection head to balance the significance of both modalities. Meanwhile, CSN leverages cross-scale attention mechanisms to mitigate the gap in receptive fields between different image levels. Additionally, we introduce a novel Local with Global Voxel Attention Encoder (LGVAE) designed to capture global semantics by extracting more comprehensive point-level information into voxel-level features. We perform comprehensive experiments on three challenging public benchmarks: KITTI, Dense and nuScenes. The results consistently demonstrate improvements across multiple 3D object detection frameworks, affirming the effectiveness and versatility of our proposed method. Remarkably, our approach achieves a substantial absolute gain of 3.1% over the baseline MVXNet on the challenging Hard set of the Dense test set.

Balanced Single-Shot Object Detection Using Cross-Context Attention-Guided Network

Cross-Modal Attentional Context Learning for RGB-D Object Detection

Composite Backbone Small Object Detection Based on Context and Multi-Scale Information with Attention Mechanism

C2BG-Net: Cross-modality and cross-scale balance network with global semantics for multi-modal 3D object detection

Object detection based on an adaptive attention mechanism

Few-Shot Object Detection Based on Adaptive Attention Mechanism and Large-Margin Softmax

Object Detection Algorithm Based on Context Information and Self-Attention Mechanism

Global Guided Cross-Modal Cross-Scale Network for RGB-D Salient Object Detection

Global contextual guided residual attention network for salient object detection

Adaptive Sparse Convolutional Networks with Global Context Enhancement for Faster Object Detection on Drone Images

Exploring multi-scale deformable context and channel-wise attention for salient object detection

CAD-Net: A Context-Aware Detection Network for Objects in Remote Sensing Imagery

Solo-to-Collaborative Dual-Attention Network for One-Shot Object Detection in Remote Sensing Images

Single-Shot Object Detector Based on Attention Mechanism.

Salient Object Detection in Optical Remote Sensing Images Based on Global Context Mixed Attention

Exploring Context Information for Accurate and Fast Object Detection

Object Detection via Aspect Ratio and Context Aware Region-based Convolutional Networks

Single-Shot Balanced Detector for Geospatial Object Detection

SAC-Net: Spatial Attenuation Context for Salient Object Detection

PosNeg-Balanced Anchors with Aligned Features for Single-Shot Object Detection

Cross-Scale Hybrid Gaussian Attention Network for Object Detection in Remote Sensing Images