Abstract:The model of autoencoder is one of the most typical deep learning models that have been mainly used in unsupervised feature learning for many applications like recognition, identification and mining. Autoencoder algorithms are compute-intensive tasks. Building large scale autoencoder model can satisfy the analysis requirement of huge volume data. But the training time sometimes becomes unbearable, which naturally leads to investigate some hardware acceleration platforms like FPGA. The software versions of autoencoder often use single-precision or double-precision expressions. But the floating point units are very expensive to implement on FPGA. Fixed-point arithmetic is often used when implementing autoencoder on hardware. But the accuracy loss is often ignored and its implications for accuracy have not been studied in previous works. There are only some works focused on accelerators using some fixed bit-widths on other neural networks models. Our work gives a comprehensive evaluation to demonstrate the fix-point precision implications on the autoencoder, achieving best performance and area efficiency. The method of data format conversion, the matrix blocking methods and the complex functions approximation are the main factors considered according to the-situation of hardware implementation. The simulation method of the data conversion, the matrix blocking with different parallelism and a simple PLA approximation method were evaluated in this paper. The results showed that the fixed-point bit-width did have effect on the performance of autoencoder. Multiple factors may have crossed effect. Each factor would have two-sided impacts for discarding the "abundant" information and the "useful" information at the same time. The representation domain must be carefully selected according to the computation parallelism. The result also showed that using fixed-point arithmetic can guarantee the precision of the autoencoder algorithm and get acceptable convergence speed.

Fixed-Point Evaluation of Extreme Learning Machine for Classification

Multiple-kernel-learning-based Extreme Learning Machine for Classification Design

Fast Sparse Approximation of Extreme Learning Machine

An experimental evaluation of extreme learning machines on several hardware devices

Benchmarking Hardware Accelerating Techniques for Extreme Learning Machine.

Extreme Learning Machine Combining Hidden-Layer Feature Weighting and Batch Training for Classification

A generic FPGA-based hardware architecture for recursive least mean p-power extreme learning machine

Fpga Implementation of Precise Convolutional Neural Network for Extreme Learning Machine

Optimizing Extreme Learning Machine Via Generalized Hebbian Learning and Intrinsic Plasticity Learning

Floating-point to Fixed-point Transformation Using Extreme Value Theory

Practical Method for ECG Classification Using Weighted ELM

Performance of the Fixed-Point Autoencoder

An Efficient Image-ELM-Based Chip Classification Algorithm.

Accuracy Evaluation of Long Short Term Memory Network Based Language Model with Fixed-Point Arithmetic.

A Fast Training Algorithm for Extreme Learning Machine Based on Matrix Decomposition.

A Low-Cost FPGA Implementation of Spiking Extreme Learning Machine with On-Chip Reward-Modulated STDP Learning

Extreme learning machine: algorithm, theory and applications

A Factorized Extreme Learning Machine and Its Applications in EEG-Based Emotion Recognition.

A study on effectiveness of extreme learning machine

A review on extreme learning machine

Fuzzy ELM for classification based on feature space