Analysis, Design and Evaluation of a High-Performance Stochastic Multilayer Perceptron: from Mini-Batch Training to Inference

Ziheng Wang,Farzad Niknia,Shanshan Liu,Pedro Reviriego,Zhen Gao,Fabrizio Lombardi
DOI: https://doi.org/10.1109/nano61778.2024.10628763
2024-01-01
Abstract:The mini-batch technique is widely used in neural network training with conventional arithmetic for its efficiency; however, its feasibility and performance in SC MLPs have rarely been studied. This paper analyzes by theory and simulation the performance of the mini-batch technique in SC MLPs; the results show that it potentially has a larger benefit in MLPs using SC than the conventional version. Moreover, a pipelined SC MLP implementation is also pursued in this paper for performing the inference process. All findings and designs provided in this paper leverage the advantages of the mini-batch technique and SC implementation to design high-performance MLPs.
What problem does this paper attempt to address?