A Weight-Reload-Eliminated Compute-in-Memory Accelerator for 60 fps 4K Super-Resolution

Yaolei Li,Jinshan Yue,Jingyu Wang,Chen Tang,Yifan He,Wenbin Jia,Kaiwei Zou,Lu Zhang,Huazhong Yang,Yongpan Liu
DOI: https://doi.org/10.1109/TCSII.2022.3221108
2023-01-01
Abstract:Convolutional neural networks (CNNs) have significantly enhanced the quality of image super-resolution (SR). However, deploying CNN with deeper layers for real-time 60fps and higher resolution SR, e.g., 4K, brings higher performance requirement, incurring huge area and power overhead as well. In this brief, a high performance compute-in-memory (CIM) accelerator with high energy and area efficiency is proposed for deep-layer CNN-based SR (CNNSR). It adopts alternate time division serial (ATS) dataflow to guarantee the high performance, which eliminates frequent and time-consuming weight reload operation of CIM macros. Moreover, considering the extremely high ratio of computation quantity to weight amount in the deep-layer CNNSR scenario, an energy and area efficiency enhanced digital CIM macro with high computation-to-memory ratio (CMR) is proposed. The proposed accelerator achieves 33.3TOPS peak performance, supporting 4K SR at no less than 60fps. It is evaluated under TSMC 28nm process,achieving 13.96TOPS/W energy efficiency and 1.85TOPS/mm(2) area efficiency, which are 2.79x and 2.2x better than the state-of-the-art CNNSR accelerator, respectively.
What problem does this paper attempt to address?