Abstract:Three-dimensional LiDAR systems that capture point cloud data enable the simultaneous acquisition of spatial geometry and multi-wavelength intensity information, thereby paving the way for three-dimensional point cloud recognition and processing. However, due to the irregular distribution, low resolution of point clouds, and limited spatial recognition accuracy in complex environments, inherent errors occur in classifying and segmenting the acquired target information. Conversely, two-dimensional visible light images provide real-color information, enabling the distinction of object contours and fine details, thus yielding clear, high-resolution images when desired. The integration of two-dimensional information with point clouds offers complementary advantages. In this paper, we present the incorporation of two-dimensional information to form a multi-modal representation. From this, we extract local features to establish three-dimensional geometric relationships and two-dimensional color relationships. We introduce a novel network model, termed MInet (Multi-Information net), which effectively captures features relating to both two-dimensional color and three-dimensional pose information. This enhanced network model improves feature saliency, thereby facilitating superior segmentation and recognition tasks. We evaluate our MInet architecture using the ShapeNet and ThreeDMatch datasets for point cloud segmentation, and the Stanford dataset for object recognition. The robust results, coupled with quantitative and qualitative experiments, demonstrate the superior performance of our proposed method in point cloud segmentation and object recognition tasks.

Deep Multi-level Feature Learning on Point Sets for 3D Object Recognition

Deep Multi-scale Learning on Point Sets for 3D Object Recognition.

Point Cloud Deep Learning Network Based on Local Domain Multi-Level Feature

Deep Neural Network for Point Sets Based on Local Feature Integration

PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space

MInet: A Novel Network Model for Point Cloud Processing by Integrating Multi-Modal Information

3D Semantic Segmentation Using Deep Learning for Large-Scale Indoor Point Cloud

Associate Semantic-Instance Segmentation of 3D Point Clouds Based on Local Feature Extraction

Object Recognition of Three-dimensional SAR Based on PointNet

Point Distance Mask Attention Network for Learning on Point Cloud

Multi-Scale PointPillars 3D Object Detection Network

Deep Learning for 3D Classification Based on Point Cloud with Local Structure

PC-Net: A Deep Network for 3D Point Clouds Analysis.

An Improved Deep Multiple-input and Single-output PointNet for 3D Model Retrieval

MFNet: Multi-Level Feature Extraction and Fusion Network for Large-Scale Point Cloud Classification

AttPNet: Attention-Based Deep Neural Network for 3D Point Set Analysis

A Novel Point Cloud Encoding Method Based on Local Information for 3D Classification and Segmentation.

3D Point Cloud Recognition Based on a Multi-View Convolutional Neural Network

PVNet: A Joint Convolutional Network of Point Cloud and Multi-View for 3D Shape Recognition.

PointpartNet: 3D point-cloud registration via deep part-based feature extraction

Point CNN:3D Face Recognition with Local Feature Descriptor and Feature Enhancement Mechanism