An analog-AI chip for energy-efficient speech recognition and transcription
S Ambrogio,P Narayanan,A Okazaki,A Fasoli,C Mackin,K Hosokawa,A Nomura,T Yasuda,A Chen,A Friz,M Ishii,J Luquin,Y Kohda,N Saulnier,K Brew,S Choi,I Ok,T Philip,V Chan,C Silvestre,I Ahsan,V Narayanan,H Tsai,G W Burr
DOI: https://doi.org/10.1038/s41586-023-06337-5
IF: 64.8
Nature
Abstract:Models of artificial intelligence (AI) that have billions of parameters can achieve high accuracy across a range of tasks1,2, but they exacerbate the poor energy efficiency of conventional general-purpose processors, such as graphics processing units or central processing units. Analog in-memory computing (analog-AI)3-7 can provide better energy efficiency by performing matrix-vector multiplications in parallel on 'memory tiles'. However, analog-AI has yet to demonstrate software-equivalent (SWeq) accuracy on models that require many such tiles and efficient communication of neural-network activations between the tiles. Here we present an analog-AI chip that combines 35 million phase-change memory devices across 34 tiles, massively parallel inter-tile communication and analog, low-power peripheral circuitry that can achieve up to 12.4 tera-operations per second per watt (TOPS/W) chip-sustained performance. We demonstrate fully end-to-end SWeq accuracy for a small keyword-spotting network and near-SWeq accuracy on the much larger MLPerf8 recurrent neural-network transducer (RNNT), with more than 45 million weights mapped onto more than 140 million phase-change memory devices across five chips.