Abstract:3D object detection is an important yet challenging problem in a myriad of vision, robotics, and human- machine interaction applications. Given an RGB-D image, the task is to infer the class labels and the 3D bounding boxes of the objects in the image. While the previous studies have made remarkable progress over the past decade, how to effectively exploit the feature fusion with neural networks for boosting 3D object detection performance remains an open problem. This paper proposes a multilevel fusion network (MFN) model to detect 3D objects in RGB-D images. The MFN model contains two streams of neural networks which respectively extracts the RGB and depth features with cascaded convolutional modules. To effectively exploit the information of 3D objects, a multilevel fusion mechanism is adopted to fuse the convolutional RGB and depth features at multiple levels. To train the network, we propose a new weighted loss function by encoding the difference of geometric attributes on 3D bounding box regression. Since the original depth data is full of noisy holes, we also develop an adaptive filtering algorithm to restore and correct the depth images. We test the proposed model on challenging RGB-D datasets. The experimental results on the datasets prove the strength and advantage of the proposed model. (c) 2021 Elsevier B.V. All rights reserved. 3D object detection is an important yet challenging problem in a myriad of vision, robotics, and human? machine interaction applications. Given an RGB-D image, the task is to infer the class labels and the 3D bounding boxes of the objects in the image. While the previous studies have made remarkable progress over the past decade, how to effectively exploit the feature fusion with neural networks for boosting 3D object detection performance remains an open problem. This paper proposes a multilevel fusion network (MFN) model to detect 3D objects in RGB-D images. The MFN model contains two streams of neural networks which respectively extracts the RGB and depth features with cascaded convolutional modules. To effectively exploit the information of 3D objects, a multilevel fusion mechanism is adopted to fuse the convolutional RGB and depth features at multiple levels. To train the network, we propose a new weighted loss function by encoding the difference of geometric attributes on 3D bounding box regression. Since the original depth data is full of noisy holes, we also develop an adaptive filtering algorithm to restore and correct the depth images. We test the proposed model on challenging RGB-D datasets. The experimental results on the datasets prove the strength and advantage of the proposed model.

MFFNet: Multimodal Feature Fusion Network for RGB-D Transparent Object Detection

MFF-Net: Towards Efficient Monocular Depth Completion With Multi-Modal Feature Fusion

MFUR-Net

Discriminative feature fusion for RGB-D salient object detection

Lightweight Multi-Level Feature Difference Fusion Network for RGB-D-T Salient Object Detection

ℱ3-Net: Feature Fusion and Filtration Network for Object Detection in Optical Remote Sensing Images

MTFNet: Mutual-Transformer Fusion Network for RGB-D Salient Object Detection

MMNet: Multi-Stage and Multi-Scale Fusion Network for RGB-D Salient Object Detection

A Deep Multimodal Feature Learning Network for RGB-D Salient Object Detection

Frustum FusionNet: Amodal 3D Object Detection with Multi-Modal Feature Fusion

Modal Complementary Fusion Network for RGB-T Salient Object Detection

A Multilevel Fusion Network for 3D Object Detection.

Illumination-Guided RGBT Object Detection With Inter- and Intra-Modality Fusion

Multi-modality information refinement fusion network for RGB-D salient object detection

Feature interaction and two-stage cross-modal fusion for RGB-D salient object detection

Modality-Induced Transfer-Fusion Network for RGB-D and RGB-T Salient Object Detection

FCMNet: Frequency-aware Cross-Modality Attention Networks for RGB-D Salient Object Detection

Unified Information Fusion Network for Multi-Modal RGB-D and RGB-T Salient Object Detection

M 2rnet: Multi-modal and Multi-Scale Refined Network for RGB-D Salient Object Detection

M2RNet: Multi-modal and Multi-scale Refined Network for RGB-D Salient Object Detection

MFFNet: Multi-modal Feature Fusion Network for V-D-T Salient Object Detection