Abstract:In previous studies, the keypoints and their feature descriptors of a hyperspectral image have not received enough attention. Compared with RGB images, the rich feature information of a hyperspectral image can not only constrain the spatial positions of the keypoints, but also provide rich spectral features for their descriptors. In this paper, we propose a deep neural network method called Hyperspectral SuperPoint Network (HSSPN) to detect the keypoints from a hyperspectral image and generate the corresponding feature descriptors. The HSSPN method contains a Shared Encoder (which is responsible for fully extracting the features of the spatial and spectral dimensions) and two decoders: the Keypoint Decoder accurately detects the keypoints according to the feature map outputted by the Shared Encoder, while the Descriptor Decoder is responsible for generating stable feature descriptors. We collected some hyperspectral images with an electronically controlled turntable, and then annotated them with a Hyperspectral Homographic Adaptation (HHA) network. The trained HSSPN gave an excellent keypoint matching performance on the test set. Hyperspectral 3D reconstruction can obtain information in a 3D space and a spectral space at the same time. Most of the previous reconstruction methods are separated for the treatments of 3D reconstruction and hyperspectral imaging, which require complicated registrations. In this study, we propose a SfM-based hyperspectral 3D reconstruction method, in which we first collect hyperspectral images from multiple viewing angles, and then apply the HSSPN method to detect the keypoints and calculate their feature descriptors. These keypoints and feature descriptors are then used to reconstruct each spectral image of the target with the SfM method. The 3D reconstruction models at these wavelengths are simply superimposed to obtain a hyperspectral 3D reconstruction model. Compared with conventional SIFT-based SfM methods, our method achieves better results in the reconstruction accuracy.

3D Keypoint Detection Based on Deep Neural Network with Sparse Autoencoder

Learning Based 3D Keypoint Detection with Local and Global Attributes in Multi-Scale Space.

SK-Net: Deep Learning on Point Cloud via End-to-end Discovery of Spatial Keypoints

3D Semantic Segmentation Using Deep Learning for Large-Scale Indoor Point Cloud

DKNAS: A Practical Deep Keypoint Extraction Framework Based on Neural Architecture Search

3D face recognition based on multiple keypoint descriptors and sparse representation.

Deep Point Convolutional Approach for 3D Model Retrieval.

Automatic Schelling Point Detection From Meshes

Spherical-Patches Extraction for Deep-Learning-Based Critical Points Detection in 3D Neuron Microscopy Images

Detecting 3D Points of Interest Using Multiple Features and Stacked Auto-encoder

A Deep Learning-Based Hyperspectral Keypoint Representation Method and Its Application for 3D Reconstruction

MSL-Net: Sharp Feature Detection Network for 3D Point Clouds

SparsePoint: Fully End-to-End Sparse 3D Object Detector

SNAKE: Shape-aware Neural 3D Keypoint Field

Detecting 3D Points of Interest Using Projective Neural Networks

Keypoint Autoencoders: Learning Interest Points of Semantics

Rethinking of learning-based 3D keypoints detection for large-scale point clouds registration

SEKD: Self-Evolving Keypoint Detection and Description

KeypointDETR: an End-to-End 3D Keypoint Detector

Learning Deep Network for Detecting 3D Object Keypoints and 6D Poses

Spiking PointNet: Spiking Neural Networks for Point Clouds