Abstract:3D object detection is an important yet challenging problem in a myriad of vision, robotics, and human- machine interaction applications. Given an RGB-D image, the task is to infer the class labels and the 3D bounding boxes of the objects in the image. While the previous studies have made remarkable progress over the past decade, how to effectively exploit the feature fusion with neural networks for boosting 3D object detection performance remains an open problem. This paper proposes a multilevel fusion network (MFN) model to detect 3D objects in RGB-D images. The MFN model contains two streams of neural networks which respectively extracts the RGB and depth features with cascaded convolutional modules. To effectively exploit the information of 3D objects, a multilevel fusion mechanism is adopted to fuse the convolutional RGB and depth features at multiple levels. To train the network, we propose a new weighted loss function by encoding the difference of geometric attributes on 3D bounding box regression. Since the original depth data is full of noisy holes, we also develop an adaptive filtering algorithm to restore and correct the depth images. We test the proposed model on challenging RGB-D datasets. The experimental results on the datasets prove the strength and advantage of the proposed model. (c) 2021 Elsevier B.V. All rights reserved. 3D object detection is an important yet challenging problem in a myriad of vision, robotics, and human? machine interaction applications. Given an RGB-D image, the task is to infer the class labels and the 3D bounding boxes of the objects in the image. While the previous studies have made remarkable progress over the past decade, how to effectively exploit the feature fusion with neural networks for boosting 3D object detection performance remains an open problem. This paper proposes a multilevel fusion network (MFN) model to detect 3D objects in RGB-D images. The MFN model contains two streams of neural networks which respectively extracts the RGB and depth features with cascaded convolutional modules. To effectively exploit the information of 3D objects, a multilevel fusion mechanism is adopted to fuse the convolutional RGB and depth features at multiple levels. To train the network, we propose a new weighted loss function by encoding the difference of geometric attributes on 3D bounding box regression. Since the original depth data is full of noisy holes, we also develop an adaptive filtering algorithm to restore and correct the depth images. We test the proposed model on challenging RGB-D datasets. The experimental results on the datasets prove the strength and advantage of the proposed model.

Multiple networks fusion approach for 3-D object recognition from 2-D view

Aerial Scene Classification Via Multilevel Fusion Based on Deep Convolutional Neural Networks.

Three-Dimensional Object Detection Network Based on Multi-Layer and Multi-Modal Fusion

Multi-View Hierarchical Fusion Network for 3D Object Retrieval and Classification

Multi-sensor Data Fusion Based on Dynamic Fuzzy Neural Network

3DMMF: 3D Object Detection Network Based on Multi-Layer and Multi-Modal Fusion

Learning high-level features by fusing multi-view representation of MLS point clouds for 3D object recognition in road environments

MFF-Net: Multimodal Feature Fusion Network for 3D Object Detection

Multi-View Adaptive Fusion Network for 3D Object Detection

One-Stage Multi-Sensor Data Fusion Convolutional Neural Network For 3d Object Detection

Multi-modal fusion network guided by prior knowledge for 3D CAD model recognition

Multi-sensor Decision-level Fusion Network Based on Attention Mechanism for Object Detection

MANet: Multimodal Attention Network based Point- View fusion for 3D Shape Recognition

Frustum FusionNet: Amodal 3D Object Detection with Multi-Modal Feature Fusion

A Multilevel Fusion Network for 3D Object Detection.

Multifeature Fusion-Based Object Detection for Intelligent Transportation Systems

Channelwise and Spatially Guided Multimodal Feature Fusion Network for 3-D Object Detection in Autonomous Vehicles

Fused DNN: A deep neural network fusion approach to fast and robust pedestrian detection

MVF-Net: A Multi-view Fusion Network for Event-based Object Classification

A method of cross-layer fusion multi-object detection and recognition based on improved faster R-CNN model in complex traffic environment

Channel-wise and Spatially-Guided Multimodal Feature Fusion Network for 3D Object Detection in Autonomous Vehicles