Abstract:In recent years, convolutional neural networks (CNNs) have gained widespread adoption in remote sensing image processing. Deploying CNN-based algorithms on satellite edge devices can alleviate the strain on data downlinks. However, CNN algorithms present challenges due to their large parameter count and high computational requirements, which conflict with the satellite platforms’ low power consumption and high real-time requirements. Moreover, remote sensing image processing tasks are diverse, requiring the platform to accommodate various network structures. To address these issues, this paper proposes an algorithm–hardware co-optimization and deployment method for FPGA-based CNN remote sensing image processing. Firstly, a series of hardware-centric model optimization techniques are proposed, including operator fusion and depth-first mapping technology, to minimize the resource overhead of CNN models. Furthermore, a versatile hardware accelerator is proposed to accelerate a wide range of commonly used CNN models after optimization. The accelerator architecture mainly consists of a parallel configurable network processing unit and a multi-level storage structure, enabling the processing of optimized networks with high throughput and low power consumption. To verify the superiority of our method, the introduced accelerator was deployed on an AMD-Xilinx VC709 evaluation board, on which the improved YOLOv2, VGG-16, and ResNet-34 networks were deployed. Experiments show that the power consumption of the accelerator is 14.97 W, and the throughput of the three networks reaches 386.74 giga operations per second (GOPS), 344.44 GOPS, and 182.34 GOPS, respectively. Comparison with related work demonstrates that the co-optimization and deployment method can accelerate remote sensing image processing CNN models and is suitable for applications in satellite edge devices.

Convolutional Neural Network Accelerator Architecture Design for Ultimate Edge Computing Scenario

A Convolutional Neural Network Accelerator Architecture with Fine-Granular Mixed Precision Configurability.

An Efficient Lightweight CNN Acceleration Architecture for Edge Computing Based-on FPGA

A High-Performance Pixel-Level Fully Pipelined Hardware Accelerator for Neural Networks

Design of FPGA Accelerator Architecture for Convolutional Neural Network

An NoC-based CNN Accelerator for Edge Computing

An Efficient FPGA-based Depthwise Separable Convolutional Neural Network Accelerator with Hardware Pruning.

FPGA-based Accelerator for Convolutional Neural Network

A Mixed-Pruning Based Framework for Embedded Convolutional Neural Network Acceleration.

Design of a Generic Dynamically Reconfigurable Convolutional Neural Network Accelerator with Optimal Balance

A High-performance Hardware Accelerator Using a Fusion Approach of Convolution and Pooling

A High Efficient Architecture for Convolution Neural Network Accelerator

A high-speed reusable quantized hardware accelerator design for CNN on constrained edge device

High Performance Convolutional Neural Network Accelerator Based on Design Space Exploration

Communication-Aware and Resource-Efficient NoC-Based Architecture for CNN Acceleration

A High Energy Efficiency and Low Resource Consumption FPGA Accelerator for Convolutional Neural Network

Algorithm–Hardware Co-Optimization and Deployment Method for Field-Programmable Gate-Array-Based Convolutional Neural Network Remote Sensing Image Processing

A Resource Efficient CNN Accelerator for Sensor Signal Processing Based on FPGA.

Throughput-Optimized FPGA Accelerator for Deep Convolutional Neural Networks

A 16.41 TOPS/W CNN Accelerator with Event-Based Layer Fusion for Real-Time Inference

A High Performance FPGA-based Accelerator for Large-Scale Convolutional Neural Networks