Abstract:Hyperspectral imaging, a rapidly evolving field, has witnessed the ascendancy of deep learning techniques, supplanting classical feature extraction and classification methods in various applications. However, many researchers employ arbitrary architectures for hyperspectral image processing, often without rigorous analysis of the interplay between spectral and spatial information. This oversight neglects the implications of combining these two modalities on model performance. In this paper, we evaluate the performance of diverse deep learning architectures for hyperspectral image segmentation. Our analysis disentangles the impact of different architectures, spanning various spectral and spatial granularities. Specifically, we investigate the effects of spectral resolution (capturing spectral information) and spatial texture (conveying spatial details) on segmentation outcomes. Additionally, we explore the transferability of knowledge from large pre-trained image foundation models, originally designed for RGB images, to the hyperspectral domain. Results show that incorporating spatial information alongside spectral data leads to improved segmentation results, and that it is essential to further work on novel architectures comprising spectral and spatial information and on the adaption of RGB foundation models into the hyperspectral domain. Furthermore, we contribute to the field by cleaning and publicly releasing the Tecnalia WEEE Hyperspectral dataset. This dataset contains different non-ferrous fractions of Waste Electrical and Electronic Equipment (WEEE), including Copper, Brass, Aluminum, Stainless Steel, and White Copper, spanning the range of 400 to 1000 nm. We expect these conclusions can guide novel researchers in the field of hyperspectral imaging.

What problem does this paper attempt to address?

The paper mainly addresses the following issues: 1. **Evaluating the performance of different deep learning architectures in hyperspectral image segmentation tasks**: The study compares various deep learning architectures (including 1D encoder-decoder using only spectral information, 2D encoder-decoder integrating spatial information, and U-Net style architectures) in handling hyperspectral images. The aim is to analyze the ability of different architectures to process spectral and spatial information and their impact on the final segmentation results. 2. **Exploring the impact of spectral resolution and spatial texture on segmentation results**: Through experiments, the paper verifies the influence of different numbers of spectral bands (from RGB to all 76 bands) and different spatial resolutions on the quality of hyperspectral image segmentation. It is found that combining spectral and spatial information can significantly improve segmentation performance. 3. **Exploring the adaptability of pre-trained base models in the hyperspectral domain**: The paper attempts to apply large-scale pre-trained models originally used for RGB images (such as DINOv2) to hyperspectral images and analyzes whether these models can effectively transfer knowledge from RGB images to hyperspectral images to improve segmentation results. 4. **Releasing and making public a new hyperspectral dataset**: The paper contributes a hyperspectral dataset containing various non-ferrous metal waste (Tecnalia WEEE Hyperspectral dataset), aiming to provide researchers in the hyperspectral imaging field with a standardized dataset for model training and performance evaluation. In summary, this paper focuses on experimentally evaluating the ability of different deep learning architectures to handle hyperspectral images, exploring how to better utilize spectral and spatial information to improve segmentation results, and providing valuable data resources for further research in this field.

Hyperspectral Dataset and Deep Learning methods for Waste from Electric and Electronic Equipment Identification (WEEE)

PCB-Vision: A Multiscene RGB-Hyperspectral Benchmark Dataset of Printed Circuit Boards

SpectralWaste Dataset: Multimodal Data for Waste Sorting Automation

Deeply Learned Broadband Encoding Stochastic Hyperspectral Imaging.

Enhancing Environmental Monitoring through Multispectral Imaging: The WasteMS Dataset for Semantic Segmentation of Lakeside Waste

Hyperspectral Benchmark: Bridging the Gap between HSI Applications through Comprehensive Dataset and Pretraining

Self-Supervised Learning Hyperspectral Image Denoiser with Separated Spectral-Spatial Feature Extraction.

Deep Learning Meets Hyperspectral Image Analysis: A Multidisciplinary Review

A Hyperspectral and RGB Dataset for Building Facade Segmentation

ZeroWaste Dataset: Towards Deformable Object Segmentation in Cluttered Scenes

Hyperspectral Image Study based on Deep Learning Model HSI-ConvNext

A Deep Learning Approach for Pixel-level Material Classification via Hyperspectral Imaging

Deep Learning for Classification of Hyperspectral Data: A Comparative Review

On combining multiscale deep learning features for the classification of hyperspectral remote sensing imagery

Deep learning-based waste detection in natural and urban environments

Optimally leveraging depth features to enhance segmentation of recyclables from cluttered construction and demolition waste streams

EWasteNet: A Two-Stream Data Efficient Image Transformer Approach for E-Waste Classification

X-ray transmission imaging of waste printed circuit boards for value estimation in recycling using machine learning

Hyperspectral Image Analysis in Single-Modal and Multimodal setting using Deep Learning Techniques

EELS hyperspectral images unmixing - using autoencoders

Deep Learning for Hyperspectral Image Classification: An Overview