Abstract:In previous studies, the keypoints and their feature descriptors of a hyperspectral image have not received enough attention. Compared with RGB images, the rich feature information of a hyperspectral image can not only constrain the spatial positions of the keypoints, but also provide rich spectral features for their descriptors. In this paper, we propose a deep neural network method called Hyperspectral SuperPoint Network (HSSPN) to detect the keypoints from a hyperspectral image and generate the corresponding feature descriptors. The HSSPN method contains a Shared Encoder (which is responsible for fully extracting the features of the spatial and spectral dimensions) and two decoders: the Keypoint Decoder accurately detects the keypoints according to the feature map outputted by the Shared Encoder, while the Descriptor Decoder is responsible for generating stable feature descriptors. We collected some hyperspectral images with an electronically controlled turntable, and then annotated them with a Hyperspectral Homographic Adaptation (HHA) network. The trained HSSPN gave an excellent keypoint matching performance on the test set. Hyperspectral 3D reconstruction can obtain information in a 3D space and a spectral space at the same time. Most of the previous reconstruction methods are separated for the treatments of 3D reconstruction and hyperspectral imaging, which require complicated registrations. In this study, we propose a SfM-based hyperspectral 3D reconstruction method, in which we first collect hyperspectral images from multiple viewing angles, and then apply the HSSPN method to detect the keypoints and calculate their feature descriptors. These keypoints and feature descriptors are then used to reconstruct each spectral image of the target with the SfM method. The 3D reconstruction models at these wavelengths are simply superimposed to obtain a hyperspectral 3D reconstruction model. Compared with conventional SIFT-based SfM methods, our method achieves better results in the reconstruction accuracy.

Unsupervised Learning of 3D Semantic Keypoints with Mutual Reconstruction.

Semantic Graph Based Place Recognition for 3D Point Clouds.

3D Reconstruction and Semantic Segmentation Method Combining PointNet and 3D-Lmnet from Single Image

Superpoint-guided Semi-supervised Semantic Segmentation of 3D Point Clouds

Understanding Pixel-Level 2D Image Semantics with 3D Keypoint Knowledge Engine

Real-Time Globally Consistent 3D Reconstruction with Semantic Priors

Semantic 3D Reconstruction with Learning MVS and 2D Segmentation of Aerial Images

Unsupervised 3D Keypoint Discovery with Multi-View Geometry

Pointwise Geometric and Semantic Learning Network on 3D Point Clouds.

SSR-2D: Semantic 3D Scene Reconstruction from 2D Images

Point Scene Understanding Via Disentangled Instance Mesh Reconstruction

Unsupervised distribution-aware keypoints generation from 3D point clouds

Geometry-semantic Aware for Monocular 3D Semantic Scene Completion

3D Face Reconstruction and Semantic Annotation from Single Depth Image

Skeleton Merger: an Unsupervised Aligned Keypoint Detector

GrowSP: Unsupervised Semantic Segmentation of 3D Point Clouds

3D Scene Reconstruction with Sparse LiDAR Data and Monocular Image in Single Frame

A Deep Learning-Based Hyperspectral Keypoint Representation Method and Its Application for 3D Reconstruction

Parallel K Nearest Neighbor Matching for 3D Reconstruction.

Keypoint Autoencoders: Learning Interest Points of Semantics

Multi-Task Joint Learning of 3D Keypoint Saliency and Correspondence Estimation