Abstract:3D object detection is an important yet challenging problem in a myriad of vision, robotics, and human- machine interaction applications. Given an RGB-D image, the task is to infer the class labels and the 3D bounding boxes of the objects in the image. While the previous studies have made remarkable progress over the past decade, how to effectively exploit the feature fusion with neural networks for boosting 3D object detection performance remains an open problem. This paper proposes a multilevel fusion network (MFN) model to detect 3D objects in RGB-D images. The MFN model contains two streams of neural networks which respectively extracts the RGB and depth features with cascaded convolutional modules. To effectively exploit the information of 3D objects, a multilevel fusion mechanism is adopted to fuse the convolutional RGB and depth features at multiple levels. To train the network, we propose a new weighted loss function by encoding the difference of geometric attributes on 3D bounding box regression. Since the original depth data is full of noisy holes, we also develop an adaptive filtering algorithm to restore and correct the depth images. We test the proposed model on challenging RGB-D datasets. The experimental results on the datasets prove the strength and advantage of the proposed model. (c) 2021 Elsevier B.V. All rights reserved. 3D object detection is an important yet challenging problem in a myriad of vision, robotics, and human? machine interaction applications. Given an RGB-D image, the task is to infer the class labels and the 3D bounding boxes of the objects in the image. While the previous studies have made remarkable progress over the past decade, how to effectively exploit the feature fusion with neural networks for boosting 3D object detection performance remains an open problem. This paper proposes a multilevel fusion network (MFN) model to detect 3D objects in RGB-D images. The MFN model contains two streams of neural networks which respectively extracts the RGB and depth features with cascaded convolutional modules. To effectively exploit the information of 3D objects, a multilevel fusion mechanism is adopted to fuse the convolutional RGB and depth features at multiple levels. To train the network, we propose a new weighted loss function by encoding the difference of geometric attributes on 3D bounding box regression. Since the original depth data is full of noisy holes, we also develop an adaptive filtering algorithm to restore and correct the depth images. We test the proposed model on challenging RGB-D datasets. The experimental results on the datasets prove the strength and advantage of the proposed model.

TBFNT3D: Two-Branch Fusion Network with Transformer for Multimodal Indoor 3D Object Detection

Image attention transformer network for indoor 3D object detection

Bridged Transformer for Vision and Point Cloud 3D Object Detection

MBDF-Net

TFIENet: Transformer Fusion Information Enhancement Network for Multi-Model 3D Object Detection

A Multilevel Fusion Network for 3D Object Detection.

Improving 3D Object Detection with Channel-wise Transformer

TFIENet: Transformer Fusion Information Enhancement Network for Multimodel 3-D Object Detection.

BMFN3D: Bidirectional multilayer fusion network for indoor 3D object detection

CT3D++: Improving 3D Object Detection with Keypoint-induced Channel-wise Transformer

A Multi-Level Semantic Fusion VoteNet for 3D Object Detection on Point Clouds

Frustum FusionNet: Amodal 3D Object Detection with Multi-Modal Feature Fusion

MVX-Net: Multimodal VoxelNet for 3D Object Detection

Light-TBFNet: RGB-D Salient Detection Based on a Lightweight Two-Branch Fusion Strategy

F-3DNet: Leveraging Inner Order of Point Clouds for 3D Object Detection

DTSSD: Dual-Channel Transformer-Based Network for Point-Based 3D Object Detection

Three-Dimensional Point Cloud Object Detection Based on Feature Fusion and Enhancement

STFormer3D: Spatio-Temporal Transformer Based 3D Object Detection for Intelligent Driving.

One-Stage Multi-Sensor Data Fusion Convolutional Neural Network For 3d Object Detection

DMSC-Net: A deep Multi-Scale context network for 3D object detection of indoor point clouds

DS-Trans: A 3D Object Detection Method Based on a Deformable Spatiotemporal Transformer for Autonomous Vehicles