Abstract:We present the updated version of the HSI-Drive dataset aimed at developing automated driving systems (ADS) using hyperspectral imaging (HSI). The v2.0 version includes new annotated images from videos recorded during winter and fall in real driving scenarios. Added to the spring and summer images included in the previous v1.1 version, the new dataset contains 752 images covering the four seasons. In this paper, we show the improvements achieved over previously published results obtained on the v1.1 dataset, showcasing the enhanced performance of models trained on the new v2.0 dataset. We also show the progress made in comprehensive scene understanding by experimenting with more capable image segmentation models. These models include new segmentation categories aimed at the identification of essential road safety objects such as the presence of vehicles and road signs, as well as highly vulnerable groups like pedestrians and cyclists. In addition, we provide evidence of the performance and robustness of the models when applied to segmenting HSI video sequences captured in various environments and conditions. Finally, for a correct assessment of the results described in this work, the constraints imposed by the processing platforms that can sensibly be deployed in vehicles for ADS must be taken into account. Thus, and although implementation details are out of the scope of this paper, we focus our research on the development of computationally efficient, lightweight ML models that can eventually operate at high throughput rates. The dataset and some examples of segmented videos are available in <a class="link-external link-https" href="https://ipaccess.ehu.eus/HSI-Drive/" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to develop more efficient and accurate scene - understanding techniques in autonomous driving systems (ADS), especially to improve the performance of image segmentation models through the use of hyperspectral imaging (HSI) technology. Specifically, the author introduced the updated version v2.0 of the HSI - Drive dataset, aiming to solve the following problems: 1. **Lack of data diversity**: The previous HSI - Drive v1.1 dataset only contained data from spring and summer, while the new v2.0 version added data from autumn and winter, thus covering all four seasons and improving data diversity and representativeness. 2. **Improvement of model performance and robustness**: By adding more diverse data, the author showed how to train more accurate and robust HSI segmentation models. These models can not only identify key objects such as road signs and vehicles, but also better handle vulnerable groups such as pedestrians and cyclists. 3. **Coping with complex environmental conditions**: The HSI - Drive v2.0 dataset contains images under different lighting conditions (such as sunny, cloudy, rainy days), different time periods (such as dawn, noon, dusk), and different road conditions. This allows researchers to evaluate the performance of models under various actual driving conditions and develop more general - purpose and robust classification systems. 4. **Computational efficiency and real - time performance**: To meet the requirements of autonomous driving systems for low latency and small memory footprint, the author focused on developing lightweight and efficient machine - learning models, ensuring that these models can run in real - time on vehicles and maintain high performance. In summary, the main goal of this paper is to promote the application of hyperspectral imaging technology in the field of autonomous driving by expanding and improving the HSI - Drive dataset, especially to achieve more comprehensive and accurate scene understanding in complex and changeable actual driving environments.

HSI-Drive v2.0: More Data for New Challenges in Scene Understanding for Autonomous Driving

On-chip Hyperspectral Image Segmentation with Fully Convolutional Networks for Scene Understanding in Autonomous Driving

Hyperspectral Imaging-Based Perception in Autonomous Driving Scenarios: Benchmarking Baseline Semantic Segmentation Models

Exploring Fully Convolutional Networks for the Segmentation of Hyperspectral Imaging Applied to Advanced Driver Assistance Systems

HS3-Bench: A Benchmark and Strong Baseline for Hyperspectral Semantic Segmentation in Driving Scenarios

A Fine-Grained Dataset and its Efficient Semantic Segmentation for Unstructured Driving Scenarios

Ithaca365: Dataset and Driving Perception under Repeated and Challenging Weather Conditions

Toward Driving Scene Understanding: A Dataset for Learning Driver Behavior and Causal Reasoning

DDD17: End-To-End DAVIS Driving Dataset

Scalability in Perception for Autonomous Driving: Waymo Open Dataset.

SynDiff-AD: Improving Semantic Segmentation and End-to-End Autonomous Driving with Synthetic Data from Latent Diffusion Models

nuScenes: A Multimodal Dataset for Autonomous Driving

BEVal: A Cross-dataset Evaluation Study of BEV Segmentation Models for Autonomous Driving

AmodalSynthDrive: A Synthetic Amodal Perception Dataset for Autonomous Driving

aiMotive Dataset: A Multimodal Dataset for Robust Autonomous Driving with Long-Range Perception

SID: Stereo Image Dataset for Autonomous Driving in Adverse Conditions

Highway Driving Dataset for Semantic Video Segmentation

Hyper-Drive: Visible-Short Wave Infrared Hyperspectral Imaging Datasets for Robots in Unstructured Environments

IDD-AW: A Benchmark for Safe and Robust Segmentation of Drive Scenes in Unstructured Traffic and Adverse Weather

HSDA: High-frequency Shuffle Data Augmentation for Bird's-Eye-View Map Segmentation