Abstract:The article discusses the use of low cost System-On-Module (SOM) platforms for the implementation of efficient hyperspectral imaging (HSI) processors for application in autonomous driving. The work addresses the challenges of shaping and deploying multiple layer fully convolutional networks (FCN) for low-latency, on-board image semantic segmentation using resource- and power-constrained processing devices. The paper describes in detail the steps followed to redesign and customize a successfully trained HSI segmentation lightweight FCN that was previously tested on a high-end heterogeneous multiprocessing system-on-chip (MPSoC) to accommodate it to the constraints imposed by a low-cost SOM. This SOM features a lower-end but much cheaper MPSoC suitable for the deployment of automatic driving systems (ADS). In particular the article reports the data- and hardware-specific quantization techniques utilized to fit the FCN into a commercial fixed-point programmable AI coprocessor IP, and proposes a full customized post-training quantization scheme to reduce computation and storage costs without compromising segmentation accuracy.

What problem does this paper attempt to address?

The main problems that this paper attempts to solve are: 1. **Efficient Deployment of Hyperspectral Image Processors**: How to achieve efficient hyperspectral image (HSI) processing on resource - and power - constrained embedded systems, especially for applications in autonomous driving systems (ADS). Specifically, the paper explores the use of low - cost System - On - Module (SOM) platforms to implement efficient HSI processors. 2. **Low - Latency On - board Image Semantic Segmentation**: How to design and deploy multi - layer fully convolutional networks (FCN) to achieve low - latency, real - time image semantic segmentation on autonomous vehicles. Due to the resource and power limitations of on - board systems, this is a challenging problem. 3. **Model Quantization and Optimization**: How to adapt a successfully trained lightweight FCN to a low - cost SOM platform through customized quantization techniques (such as data - and hardware - specific quantization methods), while maintaining segmentation accuracy. The paper details how to redesign and customize FCN to adapt to low - power and resource - constrained processing devices, and proposes a complete post - training quantization scheme to reduce computational and storage costs without affecting segmentation accuracy. ### Specific Problem Summary - **Efficient Deployment**: Research on how to deploy hyperspectral image processors on low - cost, low - power SOM platforms, especially for autonomous driving applications. - **Low - Latency Processing**: Explore how to design and optimize FCN so that it can achieve low - latency image semantic segmentation on resource - constrained on - board systems. - **Model Quantization**: Propose and verify a customized quantization scheme to ensure that the model's segmentation performance is not significantly reduced during the quantization process, so that it can run efficiently on low - cost hardware. ### Solutions To achieve the above goals, the author takes the following measures: 1. **Model Optimization**: By reducing the model depth and adjusting parameters, FCN can process the complete hyperspectral cube in a single pass, reducing the number of parameters and memory usage. 2. **Quantization Techniques**: Adopt techniques such as adaptive clipping and min - max quantization to ensure that the quantized model can maintain high segmentation accuracy. 3. **Hardware Adaptation**: In view of the characteristics of the AMD - Xilinx Kria K26 SOM platform, special hardware adaptation and optimization are carried out to ensure that the model can run efficiently on this platform. These measures together ensure the efficient deployment and operation of the model on low - cost, low - power hardware, providing a feasible solution for hyperspectral image processing in autonomous driving systems.

Rapid Deployment of Domain-specific Hyperspectral Image Processors with Application to Autonomous Driving

A Scalable Real-time Semantic Segmentation Network for Autonomous Driving

SiamOHOT: A Lightweight Dual Siamese Network for Onboard Hyperspectral Object Tracking Via Joint Spatial-Spectral Knowledge Distillation

On-chip Hyperspectral Image Segmentation with Fully Convolutional Networks for Scene Understanding in Autonomous Driving

Exploring Fully Convolutional Networks for the Segmentation of Hyperspectral Imaging Applied to Advanced Driver Assistance Systems

Real-time semantic segmentation on FPGAs for autonomous vehicles with hls4ml

HARP: Hierarchical Attention Oriented Region-Based Processing for High-Performance Computation in Vision Sensor

Deep-learning Based On-Chip Rapid Spectral Imaging with High Spatial Resolution

Implementation of a perception system for autonomous vehicles using a detection-segmentation network in SoC FPGA

Towards On-Board Panoptic Segmentation of Multispectral Satellite Images

Hyperspectral Imaging-Based Perception in Autonomous Driving Scenarios: Benchmarking Baseline Semantic Segmentation Models

Towards an Efficient CNN Inference Architecture Enabling In-Sensor Processing

Multi-Task Network Pruning and Embedded Optimization for Real-time Deployment in ADAS

High Performance Hyperspectral Image Classification using Graphics Processing Units

HyperSense: Hyperdimensional Intelligent Sensing for Energy-Efficient Sparse Data Processing

Detection-segmentation convolutional neural network for autonomous vehicle perception

On-chip Real-time Hyperspectral Imager with Full CMOS Resolution Enabled by Massively Parallel Neural Network

Q-Segment: Segmenting Images In-Sensor for Vessel-Based Medical Diagnosis

A Low-Rank CNN Architecture for Real-Time Semantic Segmentation in Visual SLAM Applications

In Defense of Pre-trained ImageNet Architectures for Real-time Semantic Segmentation of Road-driving Images

An Energy-Efficient, Unified CNN Accelerator for Real-Time Multi-Object Semantic Segmentation for Autonomous Vehicle