Abstract:The model of autoencoder is one of the most typical deep learning models that have been mainly used in unsupervised feature learning for many applications like recognition, identification and mining. Autoencoder algorithms are compute-intensive tasks. Building large scale autoencoder model can satisfy the analysis requirement of huge volume data. But the training time sometimes becomes unbearable, which naturally leads to investigate some hardware acceleration platforms like FPGA. The software versions of autoencoder often use single-precision or double-precision expressions. But the floating point units are very expensive to implement on FPGA. Fixed-point arithmetic is often used when implementing autoencoder on hardware. But the accuracy loss is often ignored and its implications for accuracy have not been studied in previous works. There are only some works focused on accelerators using some fixed bit-widths on other neural networks models. Our work gives a comprehensive evaluation to demonstrate the fix-point precision implications on the autoencoder, achieving best performance and area efficiency. The method of data format conversion, the matrix blocking methods and the complex functions approximation are the main factors considered according to the-situation of hardware implementation. The simulation method of the data conversion, the matrix blocking with different parallelism and a simple PLA approximation method were evaluated in this paper. The results showed that the fixed-point bit-width did have effect on the performance of autoencoder. Multiple factors may have crossed effect. Each factor would have two-sided impacts for discarding the "abundant" information and the "useful" information at the same time. The representation domain must be carefully selected according to the computation parallelism. The result also showed that using fixed-point arithmetic can guarantee the precision of the autoencoder algorithm and get acceptable convergence speed.

Performance of the Fixed-Point Autoencoder

A Convolutional Neural Network Accelerator Architecture with Fine-Granular Mixed Precision Configurability.

Optimizing FPGA-Based DNN Accelerator with Shared Exponential Floating-Point Format

Trainable Fixed-Point Quantization for Deep Learning Acceleration on FPGAs

Low Precision Floating Point Arithmetic for High Performance FPGA-based CNN Acceleration

Laius: An 8-Bit Fixed-Point CNN Hardware Inference Engine

FxpNet: Training a deep convolutional neural network in fixed-point representation

High-Ratio Lossy Compression: Exploring the Autoencoder to Compress Scientific Data

Fixed-Point Evaluation of Extreme Learning Machine for Classification

Accuracy vs. Efficiency: Achieving Both through FPGA-Implementation Aware Neural Architecture Search

Efficient Neural Image Decoding Via Fixed-Point Inference

An Overview of FPGA Based Deep Learning Accelerators: Challenges and Opportunities.

A neural network accelerated optimization method for FPGA

Intelligent Encoding Method and Implementation for FPGA-based PDAC

An Efficient Lossless Compression Method for Internet Search Data in Hardware Accelerators

Optimal Architecture of Floating-Point Arithmetic for Neural Network Training Processors

High-Performance FPGA-Based CNN Accelerator with Block-Floating-Point Arithmetic.

Ax-BxP: Approximate Blocked Computation for Precision-Reconfigurable Deep Neural Network Acceleration

A Fine-Grained Sparse Accelerator for Multi-Precision DNN.

A High Performance Multi-Bit-Width Booth Vector Systolic Accelerator for NAS Optimized Deep Learning Neural Networks

Synetgy: Algorithm-hardware Co-design for ConvNet Accelerators on Embedded FPGAs