Abstract:Recently, by using deep neural network based algorithms, object classification, detection and semantic segmentation solutions are significantly improved. However, one challenge for 2D image-based systems is that they cannot provide accurate 3D location information. This is critical for location sensitive applications such as autonomous driving and robot navigation. On the other hand, 3D methods, such as RGB-D and RGB-LiDAR based systems, can provide solutions that significantly improve the RGB only approaches. That is why this is an interesting research area for both industry and academia. Compared with 2D image-based systems, 3D-based systems are more complicated due to the following five reasons: 1) Data representation itself is more complicated. 3D images can be represented by point clouds, meshes, volumes. 2D images have pixel grid representations. 2) The computation and memory resource requirement is higher as an extra dimension is added. 3) Different distribution of the objects and difference in scene areas between indoor and outdoor make one unified framework hard to achieve. 4) 3D data, especially for the outdoor scenario, is sparse compared with the dense 2D images which makes the detection task more challenging. Finally, large size labelled datasets, which are extremely important for supervised based algorithms, are still under construction compared with well-built 2D datasets such as ImageNet. Based on challenges listed above, the described systems are organized by application scenarios, data representation methods and main tasks addressed. At the same time, critical 2D based systems which greatly influence the 3D ones are also introduced to show the connection between them.

3D Recognition Based on Sensor Modalities for Robotic Systems: A Survey

Deep learning for 3D object recognition: A survey

A Comprehensive Review on 3D Object Detection and 6D Pose Estimation With Deep Learning

A Survey on Deep Learning Methods for Robot Vision

3D perception from binocular vision for a low cost humanoid robot NAO

A Comprehensive Study of 3-D Vision-Based Robot Manipulation

Real-time 3D Semantic Scene Perception for Egocentric Robots with Binocular Vision

Visual Perception Framework for an Intelligent Mobile Robot

3D Object Detection for Autonomous Driving: A Survey

Robotic Vision for Human-Robot Interaction and Collaboration: A Survey and Systematic Review

Recognizing Objects In-the-wild: Where Do We Stand?

Deep Learning Based 3D Segmentation: A Survey

Bridging realities: training visuo-haptic object recognition models for robots using 3D virtual simulations

Seeing by haptic glance: reinforcement learning-based 3D object Recognition

3D Object Detection from Images for Autonomous Driving: A Survey

A Survey on Perception Methods for Human–Robot Interaction in Social Robots

Deep learning-based scene understanding for autonomous robots: a survey

3D-RPP: a Novel 3D Vision-Based Pose Perception Approach for Industrial Robots

Survey on Vision-Based 3D Object Detection Methods

A survey on 3D object detection in real time for autonomous driving

A survey of Object Classification and Detection based on 2D/3D data