A Modular Approximation Methodology for Efficient Fixed-Point Hardware Implementation of the Sigmoid Function

Zhe Pan,Zonghua Gu,Xiaohong Jiang,Guoquan Zhu,De Ma
DOI: https://doi.org/10.1109/tie.2022.3146573
IF: 7.7
2022-01-01
IEEE Transactions on Industrial Electronics
Abstract:The sigmoid function is a widely used nonlinear activation function in neural networks. In this article, we present a modular approximation methodology for efficient fixed-point hardware implementation of the sigmoid function. Our design consists of three modules: piecewise linear (PWL) approximation as the initial solution, Taylor series approximation of the exponential function, and Newton-Raphson method-based approximation as the final solution. Its modularity enables the designer to flexibly choose the most appropriate approximation method for each module separately. Performance evaluation results indicate that our work strikes an appropriate balance among the objectives of approximation accuracy, hardware resource utilization, and performance.
What problem does this paper attempt to address?